Commit graph

42112 commits

Author SHA1 Message Date
Robert Haas
b30fb56b07 postgres_fdw: Push down FULL JOINs with restriction clauses.
The previous deparsing logic wasn't smart enough to produce subqueries
when deparsing; make it smart enough to do that.  However, we only do
it that way when necessary, because it generates more complicated SQL
which will be harder for any humans reading the queries to understand.

Etsuro Fujita, reviewed by Ashutosh Bapat

Discussion: http://postgr.es/m/c449261a-b033-dc02-9254-2fe5b7044795@lab.ntt.co.jp
2017-03-16 13:34:59 -04:00
Alvaro Herrera
a3eac988c2 Fix ancient get_object_address_opf_member bug
The original coding was trying to use a TypeName as a string Value,
which doesn't work; an oversight in my commit a61fd533.  Repair.

Also, make sure we cover the broken case in the relevant test script.

Backpatch to 9.5.

Discussion: https://postgr.es/m/20170315151829.bhxsvrp75xdxhm3n@alvherre.pgsql
2017-03-16 12:51:08 -03:00
Stephen Frost
cccbddeb14 Be more careful about signed vs. unsigned char
The buildfarm has reminded me that not all systems consider char to be
signed and we need to be explicit.  Adjust the various bits of mac8.c
for what we intend, mostly using casts to unsigned char as suggested by
Tom, and adjust the tests for valid input accordingly.  Explicitly make
the hexlookup table signed as it's useful to use -1 there to indicate an
invalid value.
2017-03-16 00:13:37 -04:00
Stephen Frost
7821f7229c Clean up overly paranoid checks in mac8.c
Andres' compiler points out, quite correctly, that there's no need for
some of the overly paranoid checks which were put into mac8.c.  Remove
those, as they're useless, add some comments and make a few other minor
improvements- reduce the size of hexlookup by making it a char array
instead of an int array, and pass in the ptr location directly instead
of making hex2_to_uchar re-calculate the location based off the offset
every time.
2017-03-15 23:23:28 -04:00
Robert Haas
6977b8b7f4 Port single-page btree vacuum logic to hash indexes.
This is advantageous for hash indexes for the same reasons it's good
for btrees: it accelerates space recycling, reducing bloat.

Ashutosh Sharma, reviewed by Amit Kapila and by me.  A bit of
additional hacking by me.

Discussion: http://postgr.es/m/CAE9k0PkRSyzx8dOnokEpUi2A-RFZK72WN0h9DEMv_ut9q6bPRw@mail.gmail.com
2017-03-15 22:18:56 -04:00
Andres Freund
2038bf41c9 Specify bindir in pg_isolation_regress_installcheck.
It appears dcae5facca forgot to add it to
pg_isolation_regress_installcheck, while it was added to
pg_regress_installcheck.  It seems to so far have escaped notice,
because buildfarm animals requiring it, didn't actually use
pg_isolation_regress_installcheck anywhere - that changed with
60f826c5e6, triggering failures on narwhal and frogmouth.

I've decided to not, for now at least, backpatch this, because the
relevant invocations look quite different in the back branches.  Seems
quite possible that we'll want to backport 60f826c5e6 as a whole if
it proves stable.

Discussion: https://postgr.es/m/20170315174003.3dyl4teashdwgblh@alap3.anarazel.de
2017-03-15 19:03:29 -07:00
Peter Eisentraut
bd1827c7c9 Simplify publication/subscription tests a bit
After testing RENAME TO, rename the object back.  This reduces the merge
mess when subsequent patches add test cases before or after the rename
test.
2017-03-15 16:52:12 -04:00
Peter Eisentraut
f2a9998fb3 Fix typo 2017-03-15 16:27:07 -04:00
Peter Eisentraut
e76db009f0 Add more documentation and tests for publications
Add/correct documentation and add some tests related to how access
control around adding tables to publications works.
2017-03-15 13:52:07 -04:00
Tom Lane
b5dd50f2c0 Rewrite async-connection loop in libpqwalreceiver.c, once again.
The original coding in commit 1e8a85009 didn't use PQconnectPoll per
spec, and while the rewrite in e434ad39a is closer, it still doesn't
guarantee to wait until the socket is read-ready or write-ready (as
appropriate) before calling PQconnectPoll.  It's not clear whether
that omission is causing the continuing failures on buildfarm member
bowerbird; but given the lack of other explanations meeting the
available facts, let's tighten that up and see what happens.

An independent issue in the same loop was that it had a race condition
whereby it could clear the process's latch without having serviced an
interrupt request, causing failure to respond to a cancel while waiting
for connection (the very problem 1e8a85009 was meant to fix).

Discussion: https://postgr.es/m/7295.1489596949@sss.pgh.pa.us
2017-03-15 13:26:26 -04:00
Robert Haas
1ea60ad602 Fix failure to use clamp_row_est() for parallel joins.
Commit 0c2070cefa neglected to use
clamp_row_est() where it should have done so.

Patch by me.  Report by Amit Kapila.

Discussion: http://postgr.es/m/CAA4eK1KPm8RYa1Kun3ZmQj9pb723b-EFN70j47Pid1vn3ByquA@mail.gmail.com
2017-03-15 12:28:54 -04:00
Stephen Frost
c583234662 Bump catversion for MACADDR8
Pointed out by Robert.
2017-03-15 11:19:39 -04:00
Stephen Frost
c7a9fa399d Add support for EUI-64 MAC addresses as macaddr8
This adds in support for EUI-64 MAC addresses by adding a new data type
called 'macaddr8' (using our usual convention of indicating the number
of bytes stored).

This was largely a copy-and-paste from the macaddr data type, with
appropriate adjustments for having 8 bytes instead of 6 and adding
support for converting a provided EUI-48 (6 byte format) to the EUI-64
format.  Conversion from EUI-48 to EUI-64 inserts FFFE as the 4th and
5th bytes but does not perform the IPv6 modified EUI-64 action of
flipping the 7th bit, but we add a function to perform that specific
action for the user as it may be commonly done by users who wish to
calculate their IPv6 address based on their network prefix and 48-bit
MAC address.

Author: Haribabu Kommi, with a good bit of rework of macaddr8_in by me.
Reviewed by: Vitaly Burovoy, Kuntal Ghosh

Discussion: https://postgr.es/m/CAJrrPGcUi8ZH+KkK+=TctNQ+EfkeCEHtMU_yo1mvX8hsk_ghNQ@mail.gmail.com
2017-03-15 11:16:25 -04:00
Robert Haas
42bdaebf16 Add a regression test for snapshot too old with hash indexes.
Amit Kapila, but I changed the comment not to be a copy-and-paste of
an existing one, and instead referred to it.

Discussion: http://postgr.es/m/CAA4eK1K0UJswCRf81WwJFO4H=+ZvbmKTNhAps-NkdmHRsq1GnQ@mail.gmail.com
2017-03-15 10:53:07 -04:00
Robert Haas
d8f356e1bd Fix MB regression tests for WAL-logging of hash indexes.
Thomas Munro noted that these files still contained the now-removed
deprecation message in the expected output.

Patch by Amit Kapila

Discussion: http://postgr.es/m/CAEepm=16HW_C+MBA27aOV9t4tQBU-vf1BT_yhRUTWZVMd9bq8A@mail.gmail.com
2017-03-15 07:25:36 -04:00
Robert Haas
f7b711c8bc Cosmetic fixes for hash index write-ahead logging.
Amit Kapila.  One of these was reported by Tom Lane.

Discussion: http://postgr.es/m/5515.1489514099@sss.pgh.pa.us
2017-03-15 07:22:49 -04:00
Peter Eisentraut
aefeb68741 Allow referring to functions without arguments when unique
In DDL commands referring to an existing function, allow omitting the
argument list if the function name is unique in its schema, per SQL
standard.

This uses the same logic that the regproc type uses for finding
functions by name only.

Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-03-14 23:55:19 -04:00
Peter Eisentraut
3f6ea5fc8d doc: Remove useless elements in bibliography
Under DSSSL, bibliodiv/para elements didn't show up, but under XSLT they
do, but they are kind of useless here, so remove them.
2017-03-14 23:30:01 -04:00
Peter Eisentraut
0c068205df doc: Fix TOC generation for refentries
The XSLT stylesheets by default don't show refentries inside sections in
table of contents, which for our documentation leads to some regressions
compared to the DSSSL output.  For example, in the SPI chapter, which is
mostly refentries, you don't get any usable table of contents.  Tweak
things so it's mostly back to what it was before.
2017-03-14 23:12:18 -04:00
Andres Freund
61d0c320b5 Improve grammar / fix typos in snapbuild.c.
Author: Erik Rijkers
Discussion: https://postgr.es/m/797c6c4496a1ae49cc69e90aa768bac2@xs4all.nl
2017-03-14 17:04:36 -07:00
Robert Haas
c44c47a773 Some preliminary refactoring towards partitionwise join.
Partitionwise join proposes add a concept of child join relations,
which will have the same relationship with join relations as "other
member" relations do with base relations.  These relations will need
some but not all of the handling that we currently have for join
relations, and some but not all of the handling that we currently have
for appendrels, since they are a mix of the two.  Refactor a little
bit so that the necessary bits of logic are exposed as separate
functions.

Ashutosh Bapat, reviewed and tested by Rajkumar Raghuwanshi and
by me.

Discussion: http://postgr.es/m/CAFjFpRfqotRR6cM3sooBHMHEVdkFfAZ6PyYg4GRZsoMuW08HjQ@mail.gmail.com
2017-03-14 19:25:47 -04:00
Andres Freund
60f826c5e6 Improve isolation tests infrastructure.
Previously if a directory had both isolationtester and plain
regression tests, they couldn't be run in parallel, because they'd
access the same files/directories.  That, so far, only affected
contrib/test_decoding.

Rather than fix that locally in contrib/test_decoding, improve
pg_regress_isolation_[install]check to use separate resources from
plain regression tests.

That requires a minor change in pg_regress, namely that the
--outputdir is created if not already existing, that seems like good
idea anyway.

Use the improved helpers even where previously not used.

Author: Tom Lane and Andres Freund
Discussion: https://postgr.es/m/20170311194831.vm5ikpczq52c2drg@alap3.anarazel.de
2017-03-14 15:56:17 -07:00
Peter Eisentraut
eb4da3e380 Add option to control snapshot export to CREATE_REPLICATION_SLOT
We used to export snapshots unconditionally in CREATE_REPLICATION_SLOT
in the replication protocol, but several upcoming patches want more
control over what happens.

Suppress snapshot export in pg_recvlogical, which neither needs nor can
use the exported snapshot.  Since snapshot exporting can fail this
improves reliability.

This also paves the way for allowing the creation of replication slots
on standbys, which cannot export snapshots because they cannot allocate
new XIDs.

Author: Petr Jelinek <petr.jelinek@2ndquadrant.com>
2017-03-14 17:34:22 -04:00
Andres Freund
7150402655 amcheck: Harden tests against concurrent autovacuums.
The previous coding of the test was vulnerable against autovacuum
triggering work on one of the tables in check_btree.sql.

For the purpose of the test it's entirely sufficient to check for
locks taken by the current process, so add an appropriate restriction.
While touching the test, expand it to also check for locks on the
underlying relations, rather than just the indexes.

Reported-By: Tom Lane
Discussion: https://postgr.es/m/30354.1489434301@sss.pgh.pa.us
2017-03-14 13:07:38 -07:00
Robert Haas
bb4a39637a hash: Support WAL consistency checking.
Kuntal Ghosh, reviewed by Amit Kapila and Ashutosh Sharma, with
a few tweaks by me.

Discussion: http://postgr.es/m/CAGz5QCJLERUn_zoO0eDv6_Y_d0o4tNTMPeR7ivTLBg4rUrJdwg@mail.gmail.com
2017-03-14 14:58:56 -04:00
Robert Haas
2609e91fcf Fix regression in parallel planning against inheritance tables.
Commit 51ee6f3160 accidentally changed
the behavior around inheritance hierarchies; before, we always
considered parallel paths even for very small inheritance children,
because otherwise an inheritance hierarchy with even one small child
wouldn't be eligible for parallelism.  That exception was inadverently
removed; put it back.

In passing, also adjust the degree-of-parallelism comptuation for
index-only scans not to consider the number of heap pages fetched.
Otherwise, we'll avoid parallel index-only scans on tables that are
mostly all-visible, which isn't especially logical.

Robert Haas and Amit Kapila, per a report from Ashutosh Sharma.

Discussion: http://postgr.es/m/CAE9k0PmgSoOHRd60SHu09aRVTHRSs8s6pmyhJKWHxWw9C_x+XA@mail.gmail.com
2017-03-14 14:33:14 -04:00
Robert Haas
c11453ce0a hash: Add write-ahead logging support.
The warning about hash indexes not being write-ahead logged and their
use being discouraged has been removed.  "snapshot too old" is now
supported for tables with hash indexes.  Most importantly, barring
bugs, hash indexes will now be crash-safe and usable on standbys.

This commit doesn't yet add WAL consistency checking for hash
indexes, as we now have for other index types; a separate patch has
been submitted to cure that lack.

Amit Kapila, reviewed and slightly modified by me.  The larger patch
series of which this is a part has been reviewed and tested by Álvaro
Herrera, Ashutosh Sharma, Mark Kirkwood, Jeff Janes, and Jesper
Pedersen.

Discussion: http://postgr.es/m/CAA4eK1JOBX=YU33631Qh-XivYXtPSALh514+jR8XeD7v+K3r_Q@mail.gmail.com
2017-03-14 13:27:02 -04:00
Tom Lane
2b32ac2a59 Include port number when logging successful binding to a TCP port.
Per suggestion from Andres Freund.

Discussion: https://postgr.es/m/20170314033842.st7gifec55yigz2h@alap3.anarazel.de
2017-03-14 13:18:38 -04:00
Peter Eisentraut
100871c2d1 ecpg: Improve some variable names
based on patch from Josh Soref <jsoref@gmail.com>
2017-03-14 12:58:39 -04:00
Peter Eisentraut
a47b38c9ee Spelling fixes
From: Josh Soref <jsoref@gmail.com>
2017-03-14 12:58:39 -04:00
Peter Eisentraut
f97a028d8e Spelling fixes in code comments
From: Josh Soref <jsoref@gmail.com>
2017-03-14 12:58:39 -04:00
Tom Lane
5ed6fff6b7 Make logging about multixact wraparound protection less chatty.
The original messaging design, introduced in commit 068cfadf9, seems too
chatty now that some time has elapsed since the bug fix; most installations
will be in good shape and don't really need a reminder about this on every
postmaster start.

Hence, arrange to suppress the "wraparound protections are now enabled"
message during startup (specifically, during the TrimMultiXact() call).
The message will still appear if protection becomes effective at some
later point.

Discussion: https://postgr.es/m/17211.1489189214@sss.pgh.pa.us
2017-03-14 12:47:53 -04:00
Robert Haas
c6c099d31a Update .gitignore
Commit 42fcad68a9 failed to do this.

Michael Paquier

Discussion: http://postgr.es/m/CAB7nPqTXK9Qn8TmonPf29jNXGw_CA3fTDaRfgnbHCFYr-Tx6qw@mail.gmail.com
2017-03-14 12:17:44 -04:00
Robert Haas
87f9982034 Fix failure to mark init buffers as BM_PERMANENT.
This could result in corruption of the init fork of an unlogged index
if the ambuildempty routine for that index used shared buffers to
create the init fork, which was true for brin, gin, gist, and hash
indexes.

Patch by me, based on an earlier patch by Michael Paquier, who also
reviewed this one.  This also incorporates an idea from Artur
Zakirov.

Discussion: http://postgr.es/m/CACYUyc8yccE4xfxhqxfh_Mh38j7dRFuxfaK1p6dSNAEUakxUyQ@mail.gmail.com
2017-03-14 11:51:11 -04:00
Peter Eisentraut
37289ffdbf doc: Fix synopsis 2017-03-14 11:19:12 -04:00
Robert Haas
a82178020d Update overlooked comment for Gather Merge.
Commit 355d3993c5 probably should have
done this, but nobody noticed that it was needed.
2017-03-14 07:52:11 -04:00
Robert Haas
bce352fb46 Remove some bogus logic from create_gather_merge_plan.
This logic was adapated from create_merge_append_plan, but the two
cases aren't really analogous, because create_merge_append_plan is not
projection-capable and must therefore have a tlist identical to that
of the underlying paths.  Overwriting the tlist of Gather Merge with
whatever the underlying plan happens to produce is no good at all.

Patch by me, reviewed by Rushabh Lathia, who also reported the issue
and made an initial attempt at a fix.

Discussion: http://postgr.es/m/CA+Tgmob_-oHEOBfT9S25bjqokdqv8e8xEmh9zOY+3MPr_LmuhA@mail.gmail.com
2017-03-14 07:43:45 -04:00
Tom Lane
0c87cd003d Fix busted markup.
Oversight in commit 9ca5c8721.  Per buildfarm.
2017-03-13 19:06:28 -04:00
Alvaro Herrera
a9c074ba7e Silence unused variable compiler warning
Fallout from fcec6caafa: mark a variable in
set_tablefunc_size_estimates as used for asserts only.

Also, the planner_rte_fetch() call is pointless with assertions
disabled, so enclose it in a USE_ASSERT_CHECKING #ifdef; fix the same
problem in set_subquery_size_estimates().

First problem noted by David Rowley, whose compiler is noisier than mine
in this regard.
2017-03-13 19:02:38 -03:00
Tom Lane
895e36bb3f Add a "void *" passthrough pointer for psqlscan.l's callback functions.
The immediate motivation for this is to provide clean infrastructure
for the proposed \if...\endif patch for psql; but it seems like a good
thing to have even if that patch doesn't get in.  Previously the callback
functions could only make use of application-global state, which is a
pretty severe handicap.

For the moment, the pointer is only passed through to the get_variable
callback function.  I considered also passing it to the write_error
callback, but for now let's not.  Neither psql nor pgbench has a use
for that, and in the case of psql we'd have to invent a separate wrapper
function because we would certainly not want to change the signature of
psql_error().

Discussion: https://postgr.es/m/10108.1489418309@sss.pgh.pa.us
2017-03-13 17:14:46 -04:00
Tom Lane
1c7a66a8e9 Remove unnecessary dependency on statement_timeout in prepared_xacts test.
Rather than waiting around for statement_timeout to expire, we can just
try to take the table's lock in nowait mode.  This saves some fraction
under 4 seconds when running this test with prepared xacts available,
and it guards against timeout-expired-anyway failures on very slow
machines when prepared xacts are not available, as seen in a recent
failure on axolotl for instance.

This approach could fail if autovacuum were to take an exclusive lock
on the test table concurrently, but there's no reason for it to do so.

Since the main point here is to improve stability in the buildfarm,
back-patch to all supported branches.
2017-03-13 16:46:32 -04:00
Michael Meskes
9ca5c8721d Document two phase commit commands in ecpg docu.
Patch by Masahiko Sawada
2017-03-13 21:03:55 +01:00
Michael Meskes
42fcad68a9 Add test case for two phase commit. Also by Masahiko Sawada. 2017-03-13 21:03:55 +01:00
Michael Meskes
43c79c7d68 Ecpg should support COMMIT PREPARED and ROLLBACK PREPARED.
The problem was that "begin transaction" was issued automatically
before executing COMMIT/ROLLBACK PREPARED if not in auto commit. This fix by
Masahiko Sawada fixes this.
2017-03-13 21:03:55 +01:00
Heikki Linnakangas
dd12bef58c Include array size in forward declaration.
Some compilers require it. At least Visual Studio, according to the
buildfarm, and gcc with the -pedantic flag.
2017-03-13 21:53:38 +02:00
Peter Eisentraut
57488c1ce3 Fix compiler warning
From: David Rowley <david.rowley@2ndquadrant.com>
2017-03-13 15:44:50 -04:00
Peter Eisentraut
1e6de941e3 Change xlog to WAL in some error messages 2017-03-13 15:42:10 -04:00
Heikki Linnakangas
aeed17d000 Use radix tree for character encoding conversions.
Replace the mapping tables used to convert between UTF-8 and other
character encodings with new radix tree-based maps. Looking up an entry in
a radix tree is much faster than a binary search in the old maps. As a
bonus, the radix tree representation is also more compact, making the
binaries slightly smaller.

The "combined" maps work the same as before, with binary search. They are
much smaller than the main tables, so it doesn't matter so much. However,
the "combined" maps are now stored in the same .map files as the main
tables. This seems more clear, since they're always used together, and
generated from the same source files.

Patch by Kyotaro Horiguchi, with lot of hacking by me at various stages.
Reviewed by Michael Paquier and Daniel Gustafsson.

Discussion: https://www.postgresql.org/message-id/20170306.171609.204324917.horiguchi.kyotaro%40lab.ntt.co.jp
2017-03-13 20:46:39 +02:00
Heikki Linnakangas
84892692fd Remove obsolete references to JIS0201.TXT JIS0208.TXT.
We don't use those files anymore, since commit 1de9cc0dcc.
2017-03-13 19:06:56 +02:00
Peter Eisentraut
d69fae203c initdb: Re-add translatable string that got lost 2017-03-13 09:10:49 -04:00