Commit graph

43325 commits

Author SHA1 Message Date
Tom Lane
f1b10496a5 First-draft release notes for 9.6.5.
As usual, the release notes for other branches will be made by cutting
these down, but put them up for community review first.  Note the first
entry is only for 9.4.
2017-08-26 15:19:32 -04:00
Michael Meskes
04fbe0e451 Changed order of statements and added an additiona MSVC safeguard to make ecpg
thread test cases work on Windows.
2017-08-26 19:11:58 +02:00
Peter Eisentraut
2073c641b4 pg_test_timing: Some NLS fixes
The string "% of total" was marked by xgettext to be a c-format, but it
is actually not, so mark up the source to prevent that.

Compute the column widths of the final display dynamically based on the
translated strings, so that translations don't mess up the display
accidentally.
2017-08-26 09:25:19 -04:00
Michael Meskes
a772624b1d Make setlocale in ECPG test cases thread aware on Windows.
Fix threaded test cases on Windows not to crash in setlocale() which can be
global or local to a thread on Windows.

Author: Christian Ullrich
2017-08-26 12:57:21 +02:00
Robert Haas
449338cc64 Improve low-level backup documentation.
Our documentation hasn't really caught up with the fact that
non-exclusive backups can now be taken using pg_start_backup and
pg_stop_backup even on standbys.  Update, also correcting some
errors introduced by 52f8a59dd9.
Updates to the 9.6 documentation are needed as well, but that
will need a separate patch as some things are different on that
version.

David Steele, reviewed by Robert Haas and Michael Paquier

Discussion: http://postgr.es/m/d4d951b9-89c0-6bc1-b6ff-d0b2dd5a8966@pgmasters.net
2017-08-25 15:20:27 -04:00
Tom Lane
aae62278e1 Fix locale dependency in new ecpg test case.
Force sorting in "C" locale so that the output ordering doesn't vary,
per buildfarm.

In passing, add missing .gitignore entries.

Discussion: https://postgr.es/m/0975f4bb-5dee-c33c-b719-3ce44026d397@chrullrich.net
2017-08-25 14:17:33 -04:00
Peter Eisentraut
99ce446ada pg_upgrade: Remove more dead code
related to 6ce6a61840

Reported-by: Christoph Berg <myon@debian.org>
2017-08-25 12:02:29 -04:00
Peter Eisentraut
e86ac70d6e Message translatability fixes 2017-08-25 11:52:31 -04:00
Michael Meskes
d22e9d5305 Implement DO CONTINUE action for ECPG WHENEVER statement.
Author: Vinayak Pokale
Reviewed-By: Masahiko Sawada
2017-08-25 15:17:29 +02:00
Tom Lane
3f4c7917b3 Code review for pushing LIMIT through subqueries.
Minor improvements for commit 1f6d515a6.  We do not need the (rather
expensive) test for SRFs in the targetlist, because since v10 any
such SRFs would appear in separate ProjectSet nodes.  Also, make the
code look more like the existing cases by turning it into a simple
recursion --- the argument that there might be some performance
benefit to contorting the code seems unfounded to me, especially since
any good compiler should turn the tail-recursion into iteration anyway.

Discussion: http://postgr.es/m/CADE5jYLuugnEEUsyW6Q_4mZFYTxHxaVCQmGAsF0yiY8ZDggi-w@mail.gmail.com
2017-08-25 09:05:17 -04:00
Andres Freund
d36f7efb39 Add minimal regression test for blessed record type transfer.
Test that blessed records can be transferred through a TupleQueue and
correctly decoded by another backend.  While touching the file, make
sure that force_parallel_mode settings only cover relevant tests.

Author: Thomas Munro, editorialized by Andres Freund
Reviewed-By: Andres Freund
Discussion: https://postgr.es/m/20170823054644.efuzftxjpfi6wwqs%40alap3.anarazel.de
2017-08-24 17:42:49 -07:00
Andres Freund
d7694fc148 Consolidate the function pointer types used by dshash.c.
Commit 8c0d7bafad introduced dshash with hash
and compare functions like DynaHash's, and also variants that take a user
data pointer instead of size.  Simplify the interface by merging them into
a single pair of function pointer types that take both size and a user data
pointer.

Since it is anticipated that memcmp and tag_hash behavior will be a common
requirement, provide wrapper functions dshash_memcmp and dshash_memhash that
conform to the new function types.

Author: Thomas Munro
Reviewed-By: Andres Freund
Discussion: https://postgr.es/m/20170823054644.efuzftxjpfi6wwqs%40alap3.anarazel.de
2017-08-24 17:01:36 -07:00
Andres Freund
4569715bd6 Fix unlikely shared memory leak after failure in dshash_create().
Tidy-up for commit 8c0d7bafad, based on a
complaint from Andres Freund.

Author: Thomas Munro
Reviewed-By: Andres Freund
Discussion: https://postgr.es/m/20170823054644.efuzftxjpfi6wwqs%40alap3.anarazel.de
2017-08-24 16:58:30 -07:00
Andres Freund
20fbf25533 Fix harmless thinko in dsa.c.
Commit 16be2fd100 added DSA_ALLOC_HUGE,
DSA_ALLOC_ZERO and DSA_ALLOC_NO_OOM which have the same numerical
values and meanings as the similarly named MCXT_... macros.  In one
place we accidentally used MCXT_ALLOC_NO_OOM when DSA_ALLOC_NO_OOM is
wanted, so tidy that up.

Author: Thomas Munro
Discussion: http://postgr.es/m/CAEepm=2AimHxVkkxnMfQvbZMkXy0uKbVa0-D38c5-qwrCm4CMQ@mail.gmail.com
Backpatch: 10, where dsa was introduced.
2017-08-24 15:07:40 -07:00
Stephen Frost
0cdc3e47be psql: Fix \gx when FETCH_COUNT is used
Set expanded output when requested through \gx in ExecQueryUsingCursor()
(used when FETCH_COUNT is set).

Discussion: https://www.postgresql.org/message-id/CB7A53AA-5645-4BDD-AB07-4D22CD9D8FF1%40gmx.net
Author: Tobias Bussmann
2017-08-24 16:20:50 -04:00
Peter Eisentraut
6ce6a61840 pg_upgrade: Remove dead code
Remove code meant for upgrading to a particular version of PostgreSQL
9.0.  Since pg_upgrade only supports upgrading to the current major
version, this code is no longer useful.
2017-08-24 15:29:35 -04:00
Peter Eisentraut
fe7774144d Increase SCRAM salt length
The original value 12 was set based on RFC 5802 for SCRAM-SHA-1, but RFC
7677 for SCRAM-SHA-256 uses 16, so use that.  (This does not affect the
validity of already stored verifiers.)

Discussion: https://www.postgresql.org/message-id/flat/12cc9297-7e05-932f-d863-765e5626ead4%402ndquadrant.com
2017-08-24 14:04:28 -04:00
Tom Lane
1177ab1dab Make new regression test case parallel-safe, and improve its output.
The test case added by commit 1f6d515a6 fails on buildfarm members that
have force_parallel_mode turned on, because we currently don't report sort
performance details from worker processes back to the master.  To fix that,
just make the test table be temp rather than regular; that's a good idea
anyway to forestall any possible interference from auto-analyze.
(The restriction that workers can't access temp tables might go away
someday, but almost certainly not before the other thing gets fixed.)

Also, improve the test so that we retain as much as possible of the
EXPLAIN ANALYZE output.  This aids debugging failures, and might also
expose problems that the preceding version masked.

Discussion: http://postgr.es/m/CADE5jYLuugnEEUsyW6Q_4mZFYTxHxaVCQmGAsF0yiY8ZDggi-w@mail.gmail.com
2017-08-24 13:39:58 -04:00
Peter Eisentraut
27b89876c0 Fix up secondary expected files
for commit 237a0b87b1
2017-08-24 11:13:55 -04:00
Peter Eisentraut
6d242ee980 Update code comment for temporary replication slots
Reported-by: Alvaro Herrera <alvherre@2ndquadrant.com>
2017-08-23 14:59:25 -04:00
Peter Eisentraut
1e1b01cd16 Fix outdated comment
Author: Thomas Munro <thomas.munro@enterprisedb.com>
2017-08-23 14:19:35 -04:00
Peter Eisentraut
237a0b87b1 Improve plural handling in error message
This does not use the normal plural handling, because no numbers appear
in the actual message.
2017-08-23 13:56:59 -04:00
Peter Eisentraut
85f4d6393d Tweak some SCRAM error messages and code comments
Clarify/correct some error messages, fix up some code comments that
confused SASL and SCRAM, and other minor fixes.  No changes in
functionality.
2017-08-23 12:29:38 -04:00
Peter Eisentraut
580ddcec39 Fix translation marker
This was erroneously removed in
55a70a023c.
2017-08-23 09:56:38 -04:00
Andres Freund
8c0d7bafad Hash tables backed by DSA shared memory.
Add general purpose chaining hash tables for DSA memory.  Unlike
DynaHash in shared memory mode, these hash tables can grow as
required, and cope with being mapped into different addresses in
different backends.

There is a wide range of potential users for such a hash table, though
it's very likely the interface will need to evolve as we come to
understand the needs of different kinds of users.  E.g support for
iterators and incremental resizing is planned for later commits and
the details of the callback signatures are likely to change.

Author: Thomas Munro
Reviewed-By: John Gorman, Andres Freund, Dilip Kumar, Robert Haas
Discussion:
	https://postgr.es/m/CAEepm=3d8o8XdVwYT6O=bHKsKAM2pu2D6sV1S_=4d+jStVCE7w@mail.gmail.com
	https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-22 22:43:07 -07:00
Peter Eisentraut
7e046e6e8a pg_upgrade: Message translatability and style fixes 2017-08-22 20:32:17 -04:00
Peter Eisentraut
b5664cfd4c doc: Mention identity column feature in section on serial
Reported-by: Basil Bourque <basil.bourque@pobox.com>
2017-08-22 19:55:21 -04:00
Andres Freund
35ea75632a Refactor typcache.c's record typmod hash table.
Previously, tuple descriptors were stored in chains keyed by a fixed size
array of OIDs.  That meant there were effectively two levels of collision
chain -- one inside and one outside the hash table.  Instead, let dynahash.c
look after conflicts for us by supplying a proper hash and equal function
pair.

This is a nice cleanup on its own, but also simplifies followup
changes allowing blessed TupleDescs to be shared between backends
participating in parallel query.

Author: Thomas Munro
Reviewed-By: Andres Freund
Discussion: https://postgr.es/m/CAEepm%3D34GVhOL%2BarUx56yx7OPk7%3DqpGsv3CpO54feqjAwQKm5g%40mail.gmail.com
2017-08-22 16:11:54 -07:00
Andres Freund
0052a0243d Add a hash_combine function for mixing hash values.
This hash function is derived from Boost's function of the same name.

Author: Andres Freund, Thomas Munro
Discussion: https://postgr.es/m/CAEepm%3D3rdgjfxW4cKvJ0OEmya2-34B0qHNG1xV0vK7TGPJGMUQ%40mail.gmail.com
Discussion: https://postgr.es/m/20170731210844.3cwrkmsmbbpt4rjc%40alap3.anarazel.de
2017-08-22 15:36:49 -07:00
Peter Eisentraut
2bfd1b1ee5 Don't install ICU collation keyword variants
Users can still create them themselves.  Instead, document Unicode TR 35
collation options for ICU, so users can create all this themselves.

Reviewed-by: Peter Geoghegan <pg@bowt.ie>
2017-08-21 19:21:07 -04:00
Peter Eisentraut
51e225da30 Expand set of predefined ICU locales
Install language+region combinations even if they are not distinct from
the language's base locale.  This gives better long-term stability of
the set of predefined locales and makes the predefined locales less
implementation-dependent and more practical for users.

Reviewed-by: Peter Geoghegan <pg@bowt.ie>
2017-08-21 19:21:07 -04:00
Robert Haas
1f6d515a67 Push limit through subqueries to underlying sort, where possible.
Douglas Doole, reviewed by Ashutosh Bapat and by me.  Minor formatting
change by me.

Discussion: http://postgr.es/m/CADE5jYLuugnEEUsyW6Q_4mZFYTxHxaVCQmGAsF0yiY8ZDggi-w@mail.gmail.com
2017-08-21 14:19:44 -04:00
Robert Haas
79ccd7cbd5 pg_prewarm: Add automatic prewarm feature.
Periodically while the server is running, and at shutdown, write out a
list of blocks in shared buffers.  When the server reaches consistency
-- unfortunatey, we can't do it before that point without breaking
things -- reload those blocks into any still-unused shared buffers.

Mithun Cy and Robert Haas, reviewed and tested by Beena Emerson,
Amit Kapila, Jim Nasby, and Rafia Sabih.

Discussion: http://postgr.es/m/CAD__OugubOs1Vy7kgF6xTjmEqTR4CrGAv8w+ZbaY_+MZeitukw@mail.gmail.com
2017-08-21 14:17:39 -04:00
Noah Misch
66ed3829df Inject $(ICU_LIBS) regardless of platform.
It appeared in a conditional that excludes AIX, Cygwin and MinGW.  Give
ICU support a chance to work on those platforms.  Back-patch to v10,
where ICU support was introduced.
2017-08-20 21:22:18 -07:00
Andres Freund
c6293249dc Partially flatten struct tupleDesc so that it can be used in DSM.
TupleDesc's attributes were already stored in contiguous memory after the
struct.  Go one step further and get rid of the array of pointers to
attributes so that they can be stored in shared memory mapped at different
addresses in each backend.  This won't work for TupleDescs with contraints
and defaults, since those point to other objects, but for many purposes
only attributes are needed.

Author: Thomas Munro
Reviewed-By: Andres Freund
Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 11:19:12 -07:00
Andres Freund
2cd7084524 Change tupledesc->attrs[n] to TupleDescAttr(tupledesc, n).
This is a mechanical change in preparation for a later commit that
will change the layout of TupleDesc.  Introducing a macro to abstract
the details of where attributes are stored will allow us to change
that in separate step and revise it in future.

Author: Thomas Munro, editorialized by Andres Freund
Reviewed-By: Andres Freund
Discussion: https://postgr.es/m/CAEepm=0ZtQ-SpsgCyzzYpsXS6e=kZWqk3g5Ygn3MDV7A8dabUA@mail.gmail.com
2017-08-20 11:19:07 -07:00
Tom Lane
b1c2d76a2f Fix possible core dump in parallel restore when using a TOC list.
Commit 3eb9a5e7c unintentionally introduced an ordering dependency
into restore_toc_entries_prefork().  The existing coding of
reduce_dependencies() contains a check to skip moving a TOC entry
to the ready_list if it wasn't initially in the pending_list.
This used to suffice to prevent reduce_dependencies() from trying to
move anything into the ready_list during restore_toc_entries_prefork(),
because the pending_list stayed empty throughout that phase; but it no
longer does.  The problem doesn't manifest unless the TOC has been
reordered by SortTocFromFile, which is how I missed it in testing.

To fix, just add a test for ready_list == NULL, converting the call
with NULL from a poor man's sanity check into an explicit command
not to touch TOC items' list membership.  Clarify some of the comments
around this; in particular, note the primary purpose of the check for
pending_list membership, which is to ensure that we can't try to restore
the same item twice, in case a TOC list forces it to be restored before
its dependency count goes to zero.

Per report from Fabrízio de Royes Mello.  Back-patch to 9.3, like the
previous commit.

Discussion: https://postgr.es/m/CAFcNs+pjuv0JL_x4+=71TPUPjdLHOXA4YfT32myj_OrrZb4ohA@mail.gmail.com
2017-08-19 13:39:51 -04:00
Peter Eisentraut
24620fc52b Fix creation of ICU comments for keyword variants
It would create the comment referring to the keyword-less parent
locale.  This was broken in ddb5fdc068.
2017-08-18 23:02:28 -04:00
Robert Haas
c4b841ba6a Fix interaction of triggers, partitioning, and EXPLAIN ANALYZE.
Add a new EState member es_leaf_result_relations, so that the trigger
code knows about ResultRelInfos created by tuple routing.  Also make
sure ExplainPrintTriggers knows about partition-related
ResultRelInfos.

Etsuro Fujita, reviewed by Amit Langote

Discussion: http://postgr.es/m/57163e18-8e56-da83-337a-22f2c0008051@lab.ntt.co.jp
2017-08-18 13:01:05 -04:00
Tom Lane
a20aac890a Temporarily revert test case from a2b70c89ca.
That code patch was good as far as it went, but the associated test case
has exposed fundamental brain damage in the parallel scan mechanism,
which is going to take nontrivial work to correct.  In the interests of
getting the buildfarm back to green so that unrelated work can proceed,
let's temporarily remove the test case.
2017-08-17 18:35:14 -04:00
Robert Haas
54cde0c4c0 Don't lock tables in RelationGetPartitionDispatchInfo.
Instead, lock them in the caller using find_all_inheritors so that
they get locked in the standard order, minimizing deadlock risks.

Also in RelationGetPartitionDispatchInfo, avoid opening tables which
are not partitioned; there's no need.

Amit Langote, reviewed by Ashutosh Bapat and Amit Khandekar

Discussion: http://postgr.es/m/91b36fa1-c197-b72f-ca6e-56c593bae68c@lab.ntt.co.jp
2017-08-17 15:43:09 -04:00
Robert Haas
ecfe59e50f Refactor validation of new partitions a little bit.
Move some logic that is currently in ATExecAttachPartition to
separate functions to facilitate future code reuse.

Ashutosh Bapat and Jeevan Ladhe

Discussion: http://postgr.es/m/CA+Tgmobbnamyvii0pRdg9pp_jLHSUvq7u5SiRrVV0tEFFU58Tg@mail.gmail.com
2017-08-17 14:49:45 -04:00
Robert Haas
1e56883a52 Attempt to clarify comments related to force_parallel_mode.
Per discussion with Tom Lane.

Discussion: http://postgr.es/m/28589.1502902172@sss.pgh.pa.us
2017-08-17 14:09:14 -04:00
Tom Lane
a2b70c89ca Fix ExecReScanGatherMerge.
Not surprisingly, since it'd never ever been tested, ExecReScanGatherMerge
didn't work.  Fix it, and add a regression test case to exercise it.

Amit Kapila

Discussion: https://postgr.es/m/CAA4eK1JkByysFJNh9M349u_nNjqETuEnY_y1VUc_kJiU0bxtaQ@mail.gmail.com
2017-08-17 13:49:22 -04:00
Tom Lane
b5178c5d08 Further tweaks to compiler flags for PL/Perl on Windows.
It now emerges that we can only rely on Perl to tell us we must use
-D_USE_32BIT_TIME_T if it's Perl 5.13.4 or later.  For older versions,
revert to our previous practice of assuming we need that symbol in
all 32-bit Windows builds.  This is not ideal, but inquiring into
which compiler version Perl was built with seems far too fragile.
In any case, we had not previously had complaints about these old
Perl versions, so let's assume this is Good Enough.  (It's still
better than the situation ante commit 5a5c2feca, in that at least
the effects are confined to PL/Perl rather than the whole PG build.)

Back-patch to all supported versions, like 5a5c2feca and predecessors.

Discussion: https://postgr.es/m/CANFyU97OVQ3+Mzfmt3MhuUm5NwPU=-FtbNH5Eb7nZL9ua8=rcA@mail.gmail.com
2017-08-17 13:13:47 -04:00
Peter Eisentraut
d542859350 doc: Update RFC URLs
Consistently use the IETF HTML links instead of a random mix of
different sites and formats.  Correct one RFC number and fix one broken
link.
2017-08-17 11:47:40 -04:00
Robert Haas
79f457e53a Remove bogus line from comment.
Spotted by Tom Lane

Discussion: http://postgr.es/m/27897.1502901074@sss.pgh.pa.us
2017-08-17 11:24:30 -04:00
Tom Lane
963af96920 Add missing "static" marker.
Per pademelon.
2017-08-17 11:17:39 -04:00
Peter Eisentraut
bfe089e372 doc: Fix table column count
Author: Erik Rijkers <er@xs4all.nl>
2017-08-17 10:37:12 -04:00
Peter Eisentraut
cf851519d4 pg_dump: Support using synchronized snapshots on standbys
This became possible by commit
6c2003f8a1.  This just makes pg_dump aware
of it and updates the documentation.

Author: Petr Jelinek <petr.jelinek@2ndquadrant.com>
2017-08-16 19:46:50 -04:00