Commit graph

43182 commits

Author SHA1 Message Date
Robert Haas
ff98a5e1e4 hash: Immediately after a bucket split, try to clean the old bucket.
If it works, then we won't be storing two copies of all the tuples
that were just moved.  If not, VACUUM will still take care of it
eventually.  Per a report from AP and analysis from Amit Kapila, it
seems that a bulk load can cause splits fast enough that VACUUM won't
deal with the problem in time to prevent bloat.

Amit Kapila; I rewrote the comment.

Discussion: http://postgr.es/m/20170704105728.mwb72jebfmok2nm2@zip.com.au
2017-08-04 19:33:01 -04:00
Tom Lane
03378c4da5 First-draft release notes for 9.6.4.
As usual, the release notes for other branches will be made by cutting
these down, but put them up for community review first.
2017-08-04 18:37:22 -04:00
Peter Eisentraut
26d40ada3f Message style improvements 2017-08-04 18:31:32 -04:00
Robert Haas
620b49a16d hash: Increase the number of possible overflow bitmaps by 8x.
Per a report from AP, it's not that hard to exhaust the supply of
bitmap pages if you create a table with a hash index and then insert a
few billion rows - and then you start getting errors when you try to
insert additional rows.  In the particular case reported by AP,
there's another fix that we can make to improve recycling of overflow
pages, which is another way to avoid the error, but there may be other
cases where this problem happens and that fix won't help.  So let's
buy ourselves as much headroom as we can without rearchitecting
anything.

The comments claim that the old limit was 64GB, but it was really
only 32GB, because we didn't use all the bits in the page for bitmap
bits - only the largest power of 2 that could fit after deducting
space for the page header and so forth.  Thus, we have 4kB per page
for bitmap bits, not 8kB.  The new limit is thus actually 8 times the
old *real* limit but only 4 times the old *purported* limit.

Since this breaks on-disk compatibility, bump HASH_VERSION.  We've
already done this earlier in this release cycle, so this doesn't cause
any incremental inconvenience for people using pg_upgrade from
releases prior to v10.  However, users who use pg_upgrade to reach
10beta3 or later from 10beta2 or earlier will need to REINDEX any hash
indexes again.

Amit Kapila and Robert Haas

Discussion: http://postgr.es/m/20170704105728.mwb72jebfmok2nm2@zip.com.au
2017-08-04 16:30:32 -04:00
Tom Lane
c30f1770a9 Apply ALTER ... SET NOT NULL recursively in ALTER ... ADD PRIMARY KEY.
If you do ALTER COLUMN SET NOT NULL against an inheritance parent table,
it will recurse to mark all the child columns as NOT NULL as well.  This
is necessary for consistency: if the column is labeled NOT NULL then
reading it should never produce nulls.

However, that didn't happen in the case where ALTER ... ADD PRIMARY KEY
marks a target column NOT NULL that wasn't before.  That was questionable
from the beginning, and now Tushar Ahuja points out that it can lead to
dump/restore failures in some cases.  So let's make that case recurse too.

Although this is meant to fix a bug, it's enough of a behavioral change
that I'm pretty hesitant to back-patch, especially in view of the lack
of similar field complaints.  It doesn't seem to be too late to put it
into v10 though.

Michael Paquier, editorialized on slightly by me

Discussion: https://postgr.es/m/b8794d6a-38f0-9d7c-ad4b-e85adf860fc9@enterprisedb.com
2017-08-04 11:45:18 -04:00
Tom Lane
97d3a0b090 Disallow SSL session tickets.
We don't actually support session tickets, since we do not create an SSL
session identifier.  But it seems that OpenSSL will issue a session ticket
on-demand anyway, which will then fail when used.  This results in
reconnection failures when using ticket-aware client-side SSL libraries
(such as the Npgsql .NET driver), as reported by Shay Rojansky.

To fix, just tell OpenSSL not to issue tickets.  At some point in the
far future, we might consider enabling tickets instead.  But the security
implications of that aren't entirely clear; and besides it would have
little benefit except for very short-lived database connections, which is
Something We're Bad At anyhow.  It would take a lot of other work to get
to a point where that would really be an exciting thing to do.

While at it, also tell OpenSSL not to use a session cache.  This doesn't
really do anything, since a backend would never populate the cache anyway,
but it might gain some micro-efficiencies and/or reduce security
exposures.

Patch by me, per discussion with Heikki Linnakangas and Shay Rojansky.
Back-patch to all supported versions.

Discussion: https://postgr.es/m/CADT4RqBU8N-csyZuzaook-c795dt22Zcwg1aHWB6tfVdAkodZA@mail.gmail.com
2017-08-04 11:07:10 -04:00
Peter Eisentraut
b374481221 Further unify ROLE and USER command grammar rules
ALTER USER ... SET did not support all the syntax variants of ALTER ROLE
...  SET.  Fix that, and to avoid further deviations of this kind, unify
many the grammar rules for ROLE/USER/GROUP commands.

Reported-by: Pavel Golub <pavel@microolap.com>
2017-08-03 20:34:45 -04:00
Tom Lane
3eb9a5e7c4 Fix pg_dump/pg_restore to emit REFRESH MATERIALIZED VIEW commands last.
Because we push all ACL (i.e. GRANT/REVOKE) restore steps to the end,
materialized view refreshes were occurring while the permissions on
referenced objects were still at defaults.  This led to failures if,
say, an MV owned by user A reads from a table owned by user B, even
if B had granted the necessary privileges to A.  We've had multiple
complaints about that type of restore failure, most recently from
Jordan Gigov.

The ideal fix for this would be to start treating ACLs as dependency-
sortable objects, rather than hard-wiring anything about their dump order
(the existing approach is a messy kluge dating to commit dc0e76ca3).
But that's going to be a rather major change, and it certainly wouldn't
lead to a back-patchable fix.  As a short-term solution, convert the
existing two-pass hack (ie, normal objects then ACLs) to a three-pass hack,
ie, normal objects then ACLs then matview refreshes.  Because this happens
in RestoreArchive(), it will also fix the problem when restoring from an
existing archive-format dump.

(Note this means that if a matview refresh would have failed under the
permissions prevailing at dump time, it'll fail during restore as well.
We'll define that as user error rather than something we should try
to work around.)

To avoid performance loss in parallel restore, we need the matview
refreshes to still be parallelizable.  Hence, clean things up enough
so that both ACLs and matviews are handled by the parallel restore
infrastructure, instead of reverting back to serial restore for ACLs.
There is still a final serial step, but it shouldn't normally have to
do anything; it's only there to try to recover if we get stuck due to
some problem like unresolved circular dependencies.

Patch by me, but it owes something to an earlier attempt by Kevin Grittner.
Back-patch to 9.3 where materialized views were introduced.

Discussion: https://postgr.es/m/28572.1500912583@sss.pgh.pa.us
2017-08-03 17:36:39 -04:00
Alvaro Herrera
9a3b5d3ad0 Fix build on zlib-less environments
Commit 4d57e83816 added support for getting I/O errors out of zlib,
but it introduced a portability problem for systems without zlib.
Repair by wrapping the zlib call inside #ifdef and restore the original
code in the other branch.

This serves to illustrate the inadequacy of the zlib abstraction in
pg_backup_archiver: there is no way to call gzerror() in that
abstraction.  This means that the several places that call GZREAD and
GZWRITE are currently doing error reporting wrongly, but ENOTIME to get
it fixed before next week's release set.

Backpatch to 9.4, like the commit that introduced the problem.
2017-08-03 14:54:28 -04:00
Robert Haas
972b6ec20b Fix lock upgrade hazard in ATExecAttachPartition.
Amit Langote

Discussion: http://postgr.es/m/CAFjFpReT_kq_uwU_B8aWDxR7jNGE=P0iELycdq5oupi=xSQTOw@mail.gmail.com
2017-08-03 14:21:00 -04:00
Robert Haas
583df3b5c5 Code beautification for ATExecAttachPartition.
Amit Langote

Discussion: http://postgr.es/m/CAFjFpReT_kq_uwU_B8aWDxR7jNGE=P0iELycdq5oupi=xSQTOw@mail.gmail.com
2017-08-03 14:19:59 -04:00
Robert Haas
86705aa8c3 Allow a foreign table CHECK constraint to be initially NOT VALID.
For a table, the constraint can be considered validated immediately,
because the table must be empty.  But for a foreign table this is
not necessarily the case.

Fixes a bug in commit f27a6b15e6.

Amit Langote, with some changes by me.

Discussion: http://postgr.es/m/d2b7419f-4a71-cf86-cc99-bfd0f359a1ea@lab.ntt.co.jp
2017-08-03 13:24:48 -04:00
Robert Haas
12a34f59bf Improve ExecModifyTable comments.
Some of these comments wrongly implied that only an AFTER ROW trigger
will cause a 'wholerow' attribute to be present for a foreign table,
but a BEFORE ROW trigger can have the same effect.  Others implied
that it would always be present for a foreign table, but that's not
true either.

Etsuro Fujita and Robert Haas

Discussion: http://postgr.es/m/10026bc7-1403-ef85-9e43-c6100c1cc0e3@lab.ntt.co.jp
2017-08-03 12:47:00 -04:00
Robert Haas
610e8ebb0f Teach map_partition_varattnos to handle whole-row expressions.
Otherwise, partitioned tables with RETURNING expressions or subject
to a WITH CHECK OPTION do not work properly.

Amit Langote, reviewed by Amit Khandekar and Etsuro Fujita.  A few
comment changes by me.

Discussion: http://postgr.es/m/9a39df80-871e-6212-0684-f93c83be4097@lab.ntt.co.jp
2017-08-03 11:21:29 -04:00
Peter Eisentraut
5ff3d73813 Add new files to nls.mk and add translation markers 2017-08-02 22:45:48 -04:00
Alvaro Herrera
4d57e83816 Fix pg_dump's errno checking for zlib I/O
Some error reports were reporting strerror(errno), which for some error
conditions coming from zlib are wrong, resulting in confusing reports
such as
  pg_restore: [compress_io] could not read from input file: Success
which makes no sense.  To correctly extract the error message we need to
use gzerror(), so let's do that.

This isn't as comprehensive or as neat as I would like, but at least it
should improve things in many common cases.  The zlib abstraction in
compress_io does not seem to be applied consistently enough; we could
perhaps improve that, but it seems master-only material, not a bug fix
for back-patching.

This problem goes back all the way, but I decided to apply back to 9.4
only, because older branches don't contain commit 14ea89366 which this
change depends on.

Authors: Vladimir Kunschikov, Álvaro Herrera
Discussion: https://postgr.es/m/1498120508308.9826@infotecs.ru
2017-08-02 18:26:59 -04:00
Tom Lane
80215156f9 Add pgtcl back to the list of externally-maintained client interfaces.
FlightAware is still maintaining this, and indeed is seemingly being
more active with it than the pgtclng fork is.  List both, for the
time being anyway.

In the back branches, also back-port commit e20f679f6 and other
recent updates to the client-interfaces list.  I think these are
probably of current interest to users of back branches.  I did
not touch the list of externally maintained PLs in the back
branches, though.  Those are much more likely to be server version
sensitive, and I don't know which of these PLs work all the way back.

Discussion: https://postgr.es/m/20170730162612.1449.58796@wrigleys.postgresql.org
2017-08-02 16:55:03 -04:00
Tom Lane
9d4e566999 Remove broken and useless entry-count printing in HASH_DEBUG code.
init_htab(), with #define HASH_DEBUG, prints a bunch of hashtable
parameters.  It used to also print nentries, but commit 44ca4022f changed
that to "hash_get_num_entries(hctl)", which is wrong (the parameter should
be "hashp").

Rather than correct the coding, though, let's just remove that field from
the printout.  The table must be empty, since we just finished building
it, so expensively calculating the number of entries is rather pointless.
Moreover hash_get_num_entries makes assumptions (about not needing locks)
which we could do without in debugging code.

Noted by Choi Doo-Won in bug #14764.  Back-patch to 9.6 where the
faulty code was introduced.

Discussion: https://postgr.es/m/20170802032353.8424.12274@wrigleys.postgresql.org
2017-08-02 12:17:08 -04:00
Peter Eisentraut
cf65201833 Get a snapshot before COPY in table sync
This fixes a crash if the local table has a function index and the
function makes non-immutable calls.

Reported-by: Scott Milliken <scott@deltaex.com>
Author: Masahiko Sawada <sawada.mshk@gmail.com>
2017-08-02 11:34:42 -04:00
Tom Lane
f352f91cbf Remove duplicate setting of SSL_OP_SINGLE_DH_USE option.
Commit c0a15e07c moved the setting of OpenSSL's SSL_OP_SINGLE_DH_USE option
into a new subroutine initialize_dh(), but forgot to remove it from where
it was.  SSL_CTX_set_options() is a trivial function, amounting indeed to
just "ctx->options |= op", hence there's no reason to contort the code or
break separation of concerns to avoid calling it twice.  So separating the
DH setup from disabling of old protocol versions is a good change, but we
need to finish the job.

Noted while poking into the question of SSL session tickets.
2017-08-02 11:28:49 -04:00
Peter Eisentraut
41cefbb6db Fix OBJECT_TYPE/OBJECT_DOMAIN confusion
This doesn't have a significant impact except that now SECURITY LABEL ON
DOMAIN rejects types that are not domains.

Reported-by: 高增琦 <pgf00a@gmail.com>
2017-08-02 10:40:32 -04:00
Tom Lane
32ca22b02d Revert test case added by commit 1e165d05fe.
The buildfarm is still showing at least three distinct behaviors for
a bad locale name in CREATE COLLATION.  Although this test was helpful
for getting the error reporting code into some usable shape, it doesn't
seem worth carrying multiple expected-files in order to support the
test in perpetuity.  So pull it back out.

Discussion: https://postgr.es/m/CAKKotZS-wcDcofXDCH=sidiuajE+nqHn2CGjLLX78anyDmi3gQ@mail.gmail.com
2017-08-01 20:15:10 -04:00
Tom Lane
514f613293 Second try at getting useful errors out of newlocale/_create_locale.
The early buildfarm returns for commit 1e165d05f are pretty awful:
not only does Windows not return a useful error, but it looks like
a lot of Unix-ish platforms don't either.  Given the number of
different errnos seen so far, guess that what's really going on is
that some newlocale() implementations fail to set errno at all.
Hence, let's try zeroing errno just before newlocale() and then
if it's still zero report as though it's ENOENT.  That should cover
the Windows case too.

It's clear that we'll have to drop the regression test case, unless
we want to maintain a separate expected-file for platforms without
HAVE_LOCALE_T.  But I'll leave it there awhile longer to see if this
actually improves matters or not.

Discussion: https://postgr.es/m/CAKKotZS-wcDcofXDCH=sidiuajE+nqHn2CGjLLX78anyDmi3gQ@mail.gmail.com
2017-08-01 17:17:20 -04:00
Tom Lane
8e7537261c Suppress less info in regression tests using DROP CASCADE.
DROP CASCADE doesn't currently promise to visit dependent objects in
a fixed order, so when the regression tests use it, we typically need
to suppress the details of which objects get dropped in order to have
predictable test output.  Traditionally we've done that by setting
client_min_messages higher than NOTICE, but there's a better way:
we can "\set VERBOSITY terse" in psql.  That suppresses the DETAIL
message with the object list, but we still get the basic notice telling
how many objects were dropped.  So at least the test case can verify
that the expected number of objects were dropped.

The VERBOSITY method was already in use in a few places, but run
around and use it wherever it makes sense.

Discussion: https://postgr.es/m/10766.1501608885@sss.pgh.pa.us
2017-08-01 16:49:23 -04:00
Tom Lane
1e165d05fe Try to deliver a sane message for _create_locale() failure on Windows.
We were just printing errno, which is certainly not gonna work on
Windows.  Now, it's not entirely clear from Microsoft's documentation
whether _create_locale() adheres to standard Windows error reporting
conventions, but let's assume it does and try to map the GetLastError
result to an errno.  If this turns out not to work, probably the best
thing to do will be to assume the error is always ENOENT on Windows.

This is a longstanding bug, but given the lack of previous field
complaints, I'm not excited about back-patching it.

Per report from Murtuza Zabuawala.

Discussion: https://postgr.es/m/CAKKotZS-wcDcofXDCH=sidiuajE+nqHn2CGjLLX78anyDmi3gQ@mail.gmail.com
2017-08-01 16:11:51 -04:00
Peter Eisentraut
c1bb787046 doc: Fix typo
Author: Fabien COELHO <coelho@cri.ensmp.fr>
2017-08-01 14:37:26 -04:00
Tom Lane
f97256570f Allow creation of C/POSIX collations without depending on libc behavior.
Most of our collations code has special handling for the locale names
"C" and "POSIX", allowing those collations to be used whether or not
the system libraries think those locale names are valid, or indeed
whether said libraries even have any locale support.  But we missed
handling things that way in CREATE COLLATION.  This meant you couldn't
clone the C/POSIX collations, nor explicitly define a new collation
using those locale names, unless the libraries allow it.  That's pretty
pointless, as well as being a violation of pg_newlocale_from_collation's
API specification.

The practical effect of this change is quite limited: it allows creating
such collations even on platforms that don't HAVE_LOCALE_T, and it allows
making "POSIX" collation objects on Windows, which before this would only
let you make "C" collation objects.  Hence, even though this is a bug fix
IMO, it doesn't seem worth the trouble to back-patch.

In passing, suppress the DROP CASCADE detail messages at the end of the
collation regression test.  I'm surprised we've never been bit by
message ordering issues there.

Per report from Murtuza Zabuawala.

Discussion: https://postgr.es/m/CAKKotZS-wcDcofXDCH=sidiuajE+nqHn2CGjLLX78anyDmi3gQ@mail.gmail.com
2017-08-01 13:51:05 -04:00
Tom Lane
b21c569cea Further improve consistency of configure's program searching.
Peter Eisentraut noted that commit 40b9f1921 had broken a configure
behavior that some people might rely on: AC_CHECK_PROGS(FOO,...) will
allow the search to be overridden by specifying a value for FOO on
configure's command line or in its environment, but AC_PATH_PROGS(FOO,...)
accepts such an override only if it's an absolute path.  We had worked
around that behavior for some, but not all, of the pre-existing uses
of AC_PATH_PROGS by just skipping the macro altogether when FOO is
already set.  Let's standardize on that workaround for all uses of
AC_PATH_PROGS, new and pre-existing, by wrapping AC_PATH_PROGS in a
new macro PGAC_PATH_PROGS.  While at it, fix a deficiency of the old
workaround code by making sure we report the setting to configure's log.

Eventually I'd like to improve PGAC_PATH_PROGS so that it converts
non-absolute override inputs to absolute form, eg "PYTHON=python3"
becomes, say, PYTHON = /usr/bin/python3.  But that will take some
nontrivial coding so it doesn't seem like a thing to do in late beta.

Discussion: https://postgr.es/m/90a92a7d-938e-507a-3bd7-ecd2b4004689@2ndquadrant.com
2017-08-01 11:40:08 -04:00
Dean Rasheed
4de6216877 Comment fix for partition_rbound_cmp().
This was an oversight in d363d42.

Beena Emerson
2017-08-01 09:40:45 +01:00
Tatsuo Ishii
e662ef0f2e Fix comment.
XLByteToSeg and XLByteToPrevSeg calculate only a segment number.  The
definition of these macros were modified by commit
dfda6ebaec but the comment remain
unchanged.

Patch by Yugo Nagata. Back patched to 9.3 and beyond.
2017-08-01 08:00:11 +09:00
Peter Eisentraut
0b02e3f128 Fix typo
Author: Masahiko Sawada <sawada.mshk@gmail.com>
2017-07-31 17:22:47 -04:00
Peter Eisentraut
f40254a799 Fix typo
Author: Etsuro Fujita <fujita.etsuro@lab.ntt.co.jp>
2017-07-31 17:08:14 -04:00
Heikki Linnakangas
4427b515e6 Doc: add v10 release notes entries for the DH parameter changes. 2017-07-31 22:47:07 +03:00
Heikki Linnakangas
c0a15e07cd Always use 2048 bit DH parameters for OpenSSL ephemeral DH ciphers.
1024 bits is considered weak these days, but OpenSSL always passes 1024 as
the key length to the tmp_dh callback. All the code to handle other key
lengths is, in fact, dead.

To remedy those issues:

* Only include hard-coded 2048-bit parameters.
* Set the parameters directly with SSL_CTX_set_tmp_dh(), without the
  callback
* The name of the file containing the DH parameters is now a GUC. This
  replaces the old hardcoded "dh1024.pem" filename. (The files for other
  key lengths, dh512.pem, dh2048.pem, etc. were never actually used.)

This is not a new problem, but it doesn't seem worth the risk and churn to
backport. If you care enough about the strength of the DH parameters on
old versions, you can create custom DH parameters, with as many bits as you
wish, and put them in the "dh1024.pem" file.

Per report by Nicolas Guini and Damian Quiroga. Reviewed by Michael Paquier.

Discussion: https://www.postgresql.org/message-id/CAMxBoUyjOOautVozN6ofzym828aNrDjuCcOTcCquxjwS-L2hGQ@mail.gmail.com
2017-07-31 22:36:09 +03:00
Tom Lane
dea6ba939f Doc: specify that the minimum supported version of Perl is 5.8.3.
Previously the docs just said "5.8 or later".  Experimentation shows
that while you can build on Unix from a git checkout with 5.8.0,
compiling recent PL/Perl requires at least 5.8.1, and you won't be
able to run the TAP tests with less than 5.8.3 because that's when
they added "prove".  (I do not have any information on just what the
MSVC build scripts require.)

Since all these versions are quite ancient, let's not split hairs
in the docs, but just say that 5.8.3 is the minimum requirement.

Discussion: https://postgr.es/m/16894.1501392088@sss.pgh.pa.us
2017-07-31 13:42:48 -04:00
Tom Lane
40b9f19217 Record full paths of programs sought by "configure".
Previously we had a mix of uses of AC_CHECK_PROG[S] and AC_PATH_PROG[S].
The only difference between those macros is that the latter emits the
full path to the program it finds, eg "/usr/bin/prove", whereas the
former emits just "prove".  Let's standardize on always emitting the
full path; this is better for documentation of the build, and it might
prevent some types of failures if later build steps are done with
a different PATH setting.

I did not touch the AC_CHECK_PROG[S] calls in ax_pthread.m4 and
ax_prog_perl_modules.m4.  There seems no need to make those diverge from
upstream, since we do not record the programs sought by the former, while
the latter's call to AC_CHECK_PROG(PERL,...) will never be reached.

Discussion: https://postgr.es/m/25937.1501433410@sss.pgh.pa.us
2017-07-31 13:02:49 -04:00
Tom Lane
b4cc35fbb7 Tighten coding for non-composite case in plperl's return_next.
Coverity complained about this code's practice of using scalar variables
as single-element arrays.  While that's really just nitpicking, it probably
is more readable to declare them as arrays, so let's do that.  A more
important point is that the code was just blithely assuming that the
result tupledesc has exactly one column; if it doesn't, we'd likely get
a crash of some sort in tuplestore_putvalues.  Since the tupledesc is
manufactured outside of plperl, that seems like an uncomfortably long
chain of assumptions.  We can nail it down at little cost with a sanity
check earlier in the function.
2017-07-31 11:33:46 -04:00
Stephen Frost
d2a51e3efc Fix function comment for dumpACL()
The comment for dumpACL() got neglected when initacls and initracls were
added and the discussion of what 'racls' is wasn't very clear either.

Per complaint from Tom.
2017-07-31 10:37:08 -04:00
Tatsuo Ishii
393d47ed0f Add missing comment in postgresql.conf.
current_source requires to restart server to reflect the new
value. Per Yugo Nagata and Masahiko Sawada.

Back patched to 9.2 and beyond.
2017-07-31 11:24:51 +09:00
Tatsuo Ishii
8b015dd723 Add missing comment in postgresql.conf.
dynamic_shared_memory_type requires to restart server to reflect
the new value. Per Yugo Nagata and Masahiko Sawada.

Back pached to 9.4 and beyond.
2017-07-31 11:06:37 +09:00
Tatsuo Ishii
9fe63092b5 Add missing comment in postgresql.conf.
max_logical_replication_workers requires to restart server to reflect
the new value. Per Yugo Nagata. Minor editing by me.
2017-07-31 10:46:32 +09:00
Andres Freund
cc9f08b6b8 Move ExecProcNode from dispatch to function pointer based model.
This allows us to add stack-depth checks the first time an executor
node is called, and skip that overhead on following
calls. Additionally it yields a nice speedup.

While it'd probably have been a good idea to have that check all
along, it has become more important after the new expression
evaluation framework in b8d7f053c5 - there's no stack depth
check in common paths anymore now. We previously relied on
ExecEvalExpr() being executed somewhere.

We should move towards that model for further routines, but as this is
required for v10, it seems better to only do the necessary (which
already is quite large).

Author: Andres Freund, Tom Lane
Reported-By: Julien Rouhaud
Discussion:
    https://postgr.es/m/22833.1490390175@sss.pgh.pa.us
    https://postgr.es/m/b0af9eaa-130c-60d0-9e4e-7a135b1e0c76@dalibo.com
2017-07-30 16:18:21 -07:00
Andres Freund
d47cfef711 Move interrupt checking from ExecProcNode() to executor nodes.
In a followup commit ExecProcNode(), and especially the large switch
it contains, will largely be replaced by a function pointer directly
to the correct node. The node functions will then get invoked by a
thin inline function wrapper. To avoid having to include miscadmin.h
in headers - CHECK_FOR_INTERRUPTS() - move the interrupt checks into
the individual executor routines.

While looking through all executor nodes, I noticed a number of
arguably missing interrupt checks, add these too.

Author: Andres Freund, Tom Lane
Reviewed-By: Tom Lane
Discussion:
    https://postgr.es/m/22833.1490390175@sss.pgh.pa.us
2017-07-30 16:06:42 -07:00
Tom Lane
9dea962b3e Include publication owner's name in the output of \dRp+.
Without this, \dRp prints information that \dRp+ does not, which
seems pretty odd.

Daniel Gustafsson

Discussion: https://postgr.es/m/3641F19B-336A-431A-86CE-A80562505C5E@yesql.se
2017-07-28 17:44:48 -04:00
Tom Lane
3c163a7fc7 PL/Perl portability fix: absorb relevant -D switches from Perl.
The Perl documentation is very clear that stuff calling libperl should
be built with the compiler switches shown by Perl's $Config{ccflags}.
We'd been ignoring that up to now, and mostly getting away with it,
but recent Perl versions contain ABI compatibility cross-checks that
fail on some builds because of this omission.  In particular the
sizeof(PerlInterpreter) can come out different due to some fields being
added or removed; which means we have a live ABI hazard that we'd better
fix rather than continuing to sweep it under the rug.

However, it still seems like a bad idea to just absorb $Config{ccflags}
verbatim.  In some environments Perl was built with a different compiler
that doesn't even use the same switch syntax.  -D switch syntax is pretty
universal though, and absorbing Perl's -D switches really ought to be
enough to fix the problem.

Furthermore, Perl likes to inject stuff like -D_LARGEFILE_SOURCE and
-D_FILE_OFFSET_BITS=64 into $Config{ccflags}, which affect libc ABIs on
platforms where they're relevant.  Adopting those seems dangerous too.
It's unclear whether a build wherein Perl and Postgres have different ideas
of sizeof(off_t) etc would work, or whether anyone would care about making
it work.  But it's dead certain that having different stdio ABIs in
core Postgres and PL/Perl will not work; we've seen that movie before.
Therefore, let's also ignore -D switches for symbols beginning with
underscore.  The symbols that we actually need to import should be the ones
mentioned in perl.h's PL_bincompat_options stanza, and none of those start
with underscore, so this seems likely to work.  (If it turns out not to
work everywhere, we could consider intersecting the symbols mentioned in
PL_bincompat_options with the -D switches.  But that will be much more
complicated, so let's try this way first.)

This will need to be back-patched, but first let's see what the
buildfarm makes of it.

Ashutosh Sharma, some adjustments by me

Discussion: https://postgr.es/m/CANFyU97OVQ3+Mzfmt3MhuUm5NwPU=-FtbNH5Eb7nZL9ua8=rcA@mail.gmail.com
2017-07-28 14:25:28 -04:00
Tom Lane
bebe174bb4 PL/Perl portability fix: avoid including XSUB.h in plperl.c.
In Perl builds that define PERL_IMPLICIT_SYS, XSUB.h defines macros
that replace a whole lot of basic libc functions with Perl functions.
We can't tolerate that in plperl.c; it breaks at least PG_TRY and
probably other stuff.  The core idea of this patch is to include XSUB.h
only in the .xs files where it's really needed, and to move any code
broken by PERL_IMPLICIT_SYS out of the .xs files and into plperl.c.

The reason this hasn't been a problem before is that our build techniques
did not result in PERL_IMPLICIT_SYS appearing as a #define in PL/Perl,
even on some platforms where Perl thinks it is defined.  That's about to
change in order to fix a nasty portability issue, so we need this work to
make the code safe for that.

Rather unaccountably, the Perl people chose XSUB.h as the place to provide
the versions of the aTHX/aTHX_ macros that are needed by code that's not
explicitly aware of the MULTIPLICITY API conventions.  Hence, just removing
XSUB.h from plperl.c fails miserably.  But we can work around that by
defining PERL_NO_GET_CONTEXT (which would make the relevant stanza of
XSUB.h a no-op anyway).  As explained in perlguts.pod, that means we need
to add a "dTHX" macro call in every C function that calls a Perl API
function.  In most of them we just add this at the top; but since the
macro fetches the current Perl interpreter pointer, more care is needed
in functions that switch the active interpreter.  Lack of the macro is
easily recognized since it results in bleats about "my_perl" not being
defined.

(A nice side benefit of this is that it significantly reduces the number
of fetches of the current interpreter pointer.  On my machine, plperl.so
gets more than 10% smaller, and there's probably some performance win too.
We could reduce the number of fetches still more by decorating the code
with pTHX_/aTHX_ macros to pass the interpreter pointer around, as
explained by perlguts.pod; but that's a task for another day.)

Formatting note: pgindent seems happy to treat "dTHX;" as a declaration
so long as it's the first thing after the left brace, as we'd already
observed with respect to the similar macro "dSP;".  If you try to put
it later in a set of declarations, pgindent puts ugly extra space
around it.

Having removed XSUB.h from plperl.c, we need only move the support
functions for spi_return_next and util_elog (both of which use PG_TRY)
out of the .xs files and into plperl.c.  This seems sufficient to
avoid the known problems caused by PERL_IMPLICIT_SYS, although we
could move more code if additional issues emerge.

This will need to be back-patched, but first let's see what the
buildfarm makes of it.

Patch by me, with some help from Ashutosh Sharma

Discussion: https://postgr.es/m/CANFyU97OVQ3+Mzfmt3MhuUm5NwPU=-FtbNH5Eb7nZL9ua8=rcA@mail.gmail.com
2017-07-28 12:25:43 -04:00
Tom Lane
8d304072a2 Fix psql tab completion for CREATE USER MAPPING.
After typing CREATE USER M..., it would not fill in MAPPING FOR,
even though that was clearly intended behavior.

Jeff Janes

Discussion: https://postgr.es/m/CAMkU=1wo2iQ6jWnN=egqOb5NxEPn0PpANEtKHr3uPooQ+nYPtw@mail.gmail.com
2017-07-27 14:13:15 -04:00
Tom Lane
77cb4a1d67 Standardize describe.c's behavior for no-matching-objects a bit more.
Most functions in this file are content to print an empty table if there
are no matching objects.  In some, the behavior is to loop over all
matching objects and print a table for each one; therefore, without any
extra logic, nothing at all would be printed if no objects match.
We accept that outcome in QUIET mode, but in normal mode it seems better
to print a helpful message.  The new \dRp+ command had not gotten that
memo; fix it.

listDbRoleSettings() is out of step on this, but I think it's better for
it to print a custom message rather than an empty table, because of the
possibility that the user is confused about what the pattern arguments mean
or which is which.  The original message wording was entirely useless for
clarifying that, though, not to mention being unlike the wordings used
elsewhere.  Improve the text, and also print the messages with psql_error
as is the general custom here.

listTables() is also out in left field, but since it's such a heavily
used function, I'm hesitant to change its behavior so much as to print
an empty table rather than a custom message.  People are probably used
to getting a message.  But we can make the wording more standardized and
helpful, and print it with psql_error rather than printing to stdout.

In both listDbRoleSettings and listTables, we play dumb and emit an
empty table, not a custom message, in QUIET mode.  That was true before
and I see no need to change it.

Several of the places printing such messages risked dumping core if
no pattern string had been provided; make them more wary.  (This case
is presently unreachable in describeTableDetails; but it shouldn't be
assuming that command.c will never pass it a null.  The text search
functions would only reach the case if a database contained no text
search objects, which is also currently impossible since we pin the
built-in objects, but again it seems unwise to assume that here.)

Daniel Gustafsson, tweaked a bit by me

Discussion: https://postgr.es/m/3641F19B-336A-431A-86CE-A80562505C5E@yesql.se
2017-07-27 13:30:59 -04:00
Tom Lane
1e2f941db1 Avoid use of sprintf/snprintf in describe.c.
Most places were already using the PQExpBuffer library for constructing
variable-length strings; bring the two stragglers into line.
describeOneTSParser was living particularly dangerously since it wasn't
even using snprintf().

Daniel Gustafsson

Discussion: https://postgr.es/m/3641F19B-336A-431A-86CE-A80562505C5E@yesql.se
2017-07-27 12:12:37 -04:00
Tom Lane
b884f629dc Sync listDbRoleSettings() with the rest of the world.
listDbRoleSettings() handled its server version check randomly differently
from every other comparable function in describe.c, not only as to code
layout but also message wording.  It also leaked memory, because its
PQExpBuffer management was also unlike everyplace else (and wrong).

Also fix an error-case leak in add_tablespace_footer().

In passing, standardize the format of function header comments in
describe.c --- we usually put "/*" alone on a line.

Daniel Gustafsson, memory leak fixes by me

Discussion: https://postgr.es/m/3641F19B-336A-431A-86CE-A80562505C5E@yesql.se
2017-07-27 11:57:29 -04:00