c8h4/postgresql
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases Wiki Activity

Prevent corr() from returning the wrong results for negative correlation

Browse source 
values. The previous coding essentially assumed that x = sqrt(x*x), which
does not hold for x < 0.

Thanks to Jie Zhang at Greenplum and Gavin Sherry for reporting this
issue.
This commit is contained in:
Neil Conway 2007-09-19 22:31:48 +00:00
parent 4893eadc3c
commit bbf4fdc253
1 changed files with 2 additions and 3 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
src/backend/utils/adt/float.c
View file

@ -8,7 +8,7 @@
*
*
* IDENTIFICATION
* $PostgreSQL: pgsql/src/backend/utils/adt/float.c,v 1.150 2007/06/05 21:31:06 tgl Exp $
* $PostgreSQL: pgsql/src/backend/utils/adt/float.c,v 1.151 2007/09/19 22:31:48 neilc Exp $
*
*-------------------------------------------------------------------------
*/
@ -2274,8 +2274,7 @@ float8_corr(PG_FUNCTION_ARGS)
if (numeratorX <= 0 || numeratorY <= 0)
PG_RETURN_NULL();
PG_RETURN_FLOAT8(sqrt((numeratorXY * numeratorXY) /
(numeratorX * numeratorY)));
PG_RETURN_FLOAT8(numeratorXY / sqrt(numeratorX * numeratorY));
}
Datum