Skip to content

Commit 63c6a24

Browse files
anarazelanarazel
committed
Run catalog reindexing test from 3dbb317 serially, to avoid deadlocks.
The tests turn out to cause deadlocks in some circumstances. Fairly reproducibly so with -DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE. Some of the deadlocks may be hard to fix without disproportionate measures, but others probably should be fixed - but not in 12. We discussed removing the new tests until we can fix the issues underlying the deadlocks, but results from buildfarm animal markhor (which runs with CLOBBER_CACHE_ALWAYS) indicates that there might be a more severe, as of yet undiagnosed, issue (including on stable branches) with reindexing catalogs. The failure is: ERROR: could not read block 0 in file "base/16384/28025": read only 0 of 8192 bytes Therefore it seems advisable to keep the tests. It's not certain that running the tests in isolation removes the risk of deadlocks. It's possible that additional locks are needed to protect against a concurrent auto-analyze or such. Per discussion with Tom Lane. Discussion: https://postgr.es/m/28926.1556664156@sss.pgh.pa.us Backpatch: 9.4-, like 3dbb317
1 parent d264bb5 commit 63c6a24

File tree

6 files changed

+75
-39
lines changed

6 files changed

+75
-39
lines changed

‎src/test/regress/expected/create_index.out

Lines changed: 0 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -3071,24 +3071,6 @@ REINDEX (VERBOSE) TABLE reindex_verbose;
30713071
INFO: index "reindex_verbose_pkey" was reindexed
30723072
DROP TABLE reindex_verbose;
30733073
--
3074-
-- check that system tables can be reindexed
3075-
--
3076-
-- whole tables
3077-
REINDEX TABLE pg_class; -- mapped, non-shared, critical
3078-
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
3079-
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
3080-
REINDEX TABLE pg_database; -- mapped, shared, critical
3081-
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
3082-
-- Check that individual system indexes can be reindexed. That's a bit
3083-
-- different from the entire-table case because reindex_relation
3084-
-- treats e.g. pg_class special.
3085-
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
3086-
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
3087-
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
3088-
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
3089-
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
3090-
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical
3091-
--
30923074
-- REINDEX SCHEMA
30933075
--
30943076
REINDEX SCHEMA schema_to_reindex; -- failure, schema does not exist

‎src/test/regress/expected/reindex_catalog.out

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
--
2+
-- Check that system tables can be reindexed.
3+
--
4+
-- Note that this test currently has to run without parallel tests
5+
-- being scheduled, as currently reindex catalog tables can cause
6+
-- deadlocks:
7+
--
8+
-- * The lock upgrade between the ShareLock acquired for the reindex
9+
-- and RowExclusiveLock needed for pg_class/pg_index locks can
10+
-- trigger deadlocks.
11+
--
12+
-- * The uniqueness checks performed when reindexing a unique/primary
13+
-- key index possibly need to wait for the transaction of a
14+
-- about-to-deleted row in pg_class to commit. That can cause
15+
-- deadlocks because, in contrast to user tables, locks on catalog
16+
-- tables are routinely released before commit - therefore the lock
17+
-- held for reindexing doesn't guarantee that no running transaction
18+
-- performed modifications in the table underlying the index.
19+
-- Check reindexing of whole tables
20+
REINDEX TABLE pg_class; -- mapped, non-shared, critical
21+
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
22+
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
23+
REINDEX TABLE pg_database; -- mapped, shared, critical
24+
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
25+
-- Check that individual system indexes can be reindexed. That's a bit
26+
-- different from the entire-table case because reindex_relation
27+
-- treats e.g. pg_class special.
28+
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
29+
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
30+
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
31+
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
32+
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
33+
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical

‎src/test/regress/parallel_schedule

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,11 @@ test: create_misc create_operator create_procedure
5757
# These depend on the above two
5858
test: create_index create_view index_including
5959

60+
# ----------
61+
# Has to run in isolation, due to deadlock risk
62+
# ----------
63+
test: reindex_catalog
64+
6065
# ----------
6166
# Another group of parallel tests
6267
# ----------

‎src/test/regress/serial_schedule

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,7 @@ test: create_procedure
6767
test: create_index
6868
test: index_including
6969
test: create_view
70+
test: reindex_catalog
7071
test: create_aggregate
7172
test: create_function_3
7273
test: create_cast

‎src/test/regress/sql/create_index.sql

Lines changed: 0 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1080,27 +1080,6 @@ CREATE TABLE reindex_verbose(id integer primary key);
10801080
REINDEX (VERBOSE) TABLE reindex_verbose;
10811081
DROP TABLE reindex_verbose;
10821082

1083-
--
1084-
-- check that system tables can be reindexed
1085-
--
1086-
1087-
-- whole tables
1088-
REINDEX TABLE pg_class; -- mapped, non-shared, critical
1089-
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
1090-
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
1091-
REINDEX TABLE pg_database; -- mapped, shared, critical
1092-
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
1093-
1094-
-- Check that individual system indexes can be reindexed. That's a bit
1095-
-- different from the entire-table case because reindex_relation
1096-
-- treats e.g. pg_class special.
1097-
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
1098-
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
1099-
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
1100-
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
1101-
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
1102-
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical
1103-
11041083
--
11051084
-- REINDEX SCHEMA
11061085
--

‎src/test/regress/sql/reindex_catalog.sql

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
--
2+
-- Check that system tables can be reindexed.
3+
--
4+
-- Note that this test currently has to run without parallel tests
5+
-- being scheduled, as currently reindex catalog tables can cause
6+
-- deadlocks:
7+
--
8+
-- * The lock upgrade between the ShareLock acquired for the reindex
9+
-- and RowExclusiveLock needed for pg_class/pg_index locks can
10+
-- trigger deadlocks.
11+
--
12+
-- * The uniqueness checks performed when reindexing a unique/primary
13+
-- key index possibly need to wait for the transaction of a
14+
-- about-to-deleted row in pg_class to commit. That can cause
15+
-- deadlocks because, in contrast to user tables, locks on catalog
16+
-- tables are routinely released before commit - therefore the lock
17+
-- held for reindexing doesn't guarantee that no running transaction
18+
-- performed modifications in the table underlying the index.
19+
20+
21+
-- Check reindexing of whole tables
22+
REINDEX TABLE pg_class; -- mapped, non-shared, critical
23+
REINDEX TABLE pg_index; -- non-mapped, non-shared, critical
24+
REINDEX TABLE pg_operator; -- non-mapped, non-shared, critical
25+
REINDEX TABLE pg_database; -- mapped, shared, critical
26+
REINDEX TABLE pg_shdescription; -- mapped, shared non-critical
27+
28+
-- Check that individual system indexes can be reindexed. That's a bit
29+
-- different from the entire-table case because reindex_relation
30+
-- treats e.g. pg_class special.
31+
REINDEX INDEX pg_class_oid_index; -- mapped, non-shared, critical
32+
REINDEX INDEX pg_class_relname_nsp_index; -- mapped, non-shared, non-critical
33+
REINDEX INDEX pg_index_indexrelid_index; -- non-mapped, non-shared, critical
34+
REINDEX INDEX pg_index_indrelid_index; -- non-mapped, non-shared, non-critical
35+
REINDEX INDEX pg_database_oid_index; -- mapped, shared, critical
36+
REINDEX INDEX pg_shdescription_o_c_index; -- mapped, shared, non-critical

0 commit comments

Comments
 (0)