Hi,
I found this bug:
if you define a plpythonu function that returns a unicode python string
the server process crashes calling that function.

The example below refers to a PostgreSQL 8.1devel installation
downloaded today from nightly snapshot but I found the same problem on
PostgreSQL 8.0.1.

Bye
Zac


Example:
OS: SuSE Linux 9.1
Python version: Python 2.3.3

test=# select version();
version
----------------------------------------------------------------------------------------
PostgreSQL 8.1devel on i686-pc-linux-gnu, compiled by GCC gcc (GCC)
3.3.3 (SuSE Linux)
(1 row)

test=# CREATE FUNCTION test_unicode() RETURNS text AS
test-# $$
test$# return u'\xe0'
test$# $$ LANGUAGE plpythonu;
CREATE FUNCTION
test=# SELECT test_unicode();
server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
The connection to the server was lost. Attempting reset: Failed.
!>



Server log (debug level 5):

DEBUG: invoking IpcMemoryCreate(size=10387456)
DEBUG: max_safe_fds = 985, usable_fds = 1019, already_open = 5
LOG: database system was shut down at 2005-06-15 14:05:00 CEST
LOG: checkpoint record is at 0/359FE4
LOG: redo record is at 0/359FE4; undo record is at 0/0; shutdown TRUE
LOG: next transaction ID: 638; next OID: 24578
LOG: next MultiXactId: 1; next MultiXactOffset: 0
LOG: database system is ready
LOG: transaction ID wrap limit is 2147484134, limited by database "test"
DEBUG: proc_exit(0)
DEBUG: shmem_exit(0)
DEBUG: exit(0)
DEBUG: reaping dead processes
LOG: connection received: host=[local] port=
DEBUG: forked new backend, pid=7842 socket=6
LOG: connection authorized: user=postgres database=test
DEBUG: postmaster child[7842]: starting with (
DEBUG: postgres
DEBUG: -v196608
DEBUG: -p
DEBUG: test
DEBUG: )
DEBUG: InitPostgres
DEBUG: StartTransaction
DEBUG: name: unnamed; blockState: DEFAULT; state: INPROGR,
xid/subid/cid: 638/1/0, nestlvl: 1, children: <>
DEBUG: CommitTransaction
DEBUG: name: unnamed; blockState: STARTED; state: INPROGR,
xid/subid/cid: 638/1/0, nestlvl: 1, children: <>
DEBUG: StartTransactionCommand
DEBUG: StartTransaction
DEBUG: name: unnamed; blockState: DEFAULT; state: INPROGR,
xid/subid/cid: 639/1/0, nestlvl: 1, children: <>
LOG: statement: select version();
DEBUG: parse tree:
DETAIL: {QUERY :commandType 1 :querySource 0 :canSetTag true
:utilityStmt <>
:resultRelation 0 :into <> :hasAggs false :hasSubLinks false
:rtable <>
:jointree {FROMEXPR :fromlist <> :quals <>} :rowMarks <>
:forUpdate false
:targetList ({TARGETENTRY :expr {FUNCEXPR :funcid 89
:funcresulttype 25
:funcretset false :funcformat 0 :args <>} :resno 1 :resname version
:ressortgroupref 0 :resorigtbl 0 :resorigcol 0 :resjunk false})
:groupClause
<> :havingQual <> :distinctClause <> :sortClause <> :limitOffset <>
:limitCount <> :setOperations <> :resultRelations <>}

DEBUG: rewritten parse tree:
DETAIL: ({QUERY :commandType 1 :querySource 0 :canSetTag true
:utilityStmt <>
:resultRelation 0 :into <> :hasAggs false :hasSubLinks false
:rtable <>
:jointree {FROMEXPR :fromlist <> :quals <>} :rowMarks <>
:forUpdate false
:targetList ({TARGETENTRY :expr {FUNCEXPR :funcid 89
:funcresulttype 25
:funcretset false :funcformat 0 :args <>} :resno 1 :resname version
:ressortgroupref 0 :resorigtbl 0 :resorigcol 0 :resjunk false})
:groupClause
<> :havingQual <> :distinctClause <> :sortClause <> :limitOffset <>
:limitCount <> :setOperations <> :resultRelations <>})

DEBUG: plan:
DETAIL: {RESULT :startup_cost 0.00 :total_cost 0.01 lan_rows 1
lan_width 0
:targetlist ({TARGETENTRY :expr {FUNCEXPR :funcid 89
:funcresulttype 25
:funcretset false :funcformat 0 :args <>} :resno 1 :resname version
:ressortgroupref 0 :resorigtbl 0 :resorigcol 0 :resjunk false})
:qual <>
:lefttree <> :righttree <> :initPlan <> :extParam (b) :allParam (b)
:nParamExec 0 :resconstantqual <>}

DEBUG: CommitTransactionCommand
DEBUG: CommitTransaction
DEBUG: name: unnamed; blockState: STARTED; state: INPROGR,
xid/subid/cid: 639/1/0, nestlvl: 1, children: <>
DEBUG: checkpoint starting
DEBUG: checkpoint complete; 0 transaction log file(s) added, 0 removed,
0 recycled
DEBUG: StartTransactionCommand
DEBUG: StartTransaction
DEBUG: name: unnamed; blockState: DEFAULT; state: INPROGR,
xid/subid/cid: 640/1/0, nestlvl: 1, children: <>
LOG: statement: CREATE FUNCTION test_unicode() RETURNS text AS
$$
return u'\xe0'
$$ LANGUAGE plpythonu;
DEBUG: parse tree:
DETAIL: {QUERY :commandType 5 :querySource 0 :canSetTag true :utilityStmt ?
:resultRelation 0 :into <> :hasAggs false :hasSubLinks false
:rtable <>
:jointree <> :rowMarks <> :forUpdate false :targetList <>
:groupClause <>
:havingQual <> :distinctClause <> :sortClause <> :limitOffset
<> :limitCount
<> :setOperations <> :resultRelations <>}

DEBUG: rewritten parse tree:
DETAIL: ({QUERY :commandType 5 :querySource 0 :canSetTag true
:utilityStmt ?
:resultRelation 0 :into <> :hasAggs false :hasSubLinks false
:rtable <>
:jointree <> :rowMarks <> :forUpdate false :targetList <>
:groupClause <>
:havingQual <> :distinctClause <> :sortClause <> :limitOffset
<> :limitCount
<> :setOperations <> :resultRelations <>})

DEBUG: ProcessUtility
DEBUG: CommitTransactionCommand
DEBUG: CommitTransaction
DEBUG: name: unnamed; blockState: STARTED; state: INPROGR,
xid/subid/cid: 640/1/0, nestlvl: 1, children: <>
DEBUG: StartTransactionCommand
DEBUG: StartTransaction
DEBUG: name: unnamed; blockState: DEFAULT; state: INPROGR,
xid/subid/cid: 641/1/0, nestlvl: 1, children: <>
LOG: statement: SELECT test_unicode();
DEBUG: parse tree:
DETAIL: {QUERY :commandType 1 :querySource 0 :canSetTag true
:utilityStmt <>
:resultRelation 0 :into <> :hasAggs false :hasSubLinks false
:rtable <>
:jointree {FROMEXPR :fromlist <> :quals <>} :rowMarks <>
:forUpdate false
:targetList ({TARGETENTRY :expr {FUNCEXPR :funcid 24578
:funcresulttype 25
:funcretset false :funcformat 0 :args <>} :resno 1 :resname
test_unicode
:ressortgroupref 0 :resorigtbl 0 :resorigcol 0 :resjunk false})
:groupClause
<> :havingQual <> :distinctClause <> :sortClause <> :limitOffset <>
:limitCount <> :setOperations <> :resultRelations <>}

DEBUG: rewritten parse tree:
DETAIL: ({QUERY :commandType 1 :querySource 0 :canSetTag true
:utilityStmt <>
:resultRelation 0 :into <> :hasAggs false :hasSubLinks false
:rtable <>
:jointree {FROMEXPR :fromlist <> :quals <>} :rowMarks <>
:forUpdate false
:targetList ({TARGETENTRY :expr {FUNCEXPR :funcid 24578
:funcresulttype 25
:funcretset false :funcformat 0 :args <>} :resno 1 :resname
test_unicode
:ressortgroupref 0 :resorigtbl 0 :resorigcol 0 :resjunk false})
:groupClause
<> :havingQual <> :distinctClause <> :sortClause <> :limitOffset <>
:limitCount <> :setOperations <> :resultRelations <>})

DEBUG: plan:
DETAIL: {RESULT :startup_cost 0.00 :total_cost 0.01 lan_rows 1
lan_width 0
:targetlist ({TARGETENTRY :expr {FUNCEXPR :funcid 24578
:funcresulttype 25
:funcretset false :funcformat 0 :args <>} :resno 1 :resname
test_unicode
:ressortgroupref 0 :resorigtbl 0 :resorigcol 0 :resjunk false})
:qual <>
:lefttree <> :righttree <> :initPlan <> :extParam (b) :allParam (b)
:nParamExec 0 :resconstantqual <>}

DEBUG: reaping dead processes
DEBUG: server process (PID 7842) was terminated by signal 11
LOG: server process (PID 7842) was terminated by signal 11
LOG: terminating any other active server processes
DEBUG: sending SIGQUIT to process 7396
DEBUG: sending SIGQUIT to process 7397
LOG: connection received: host=[local] port=
FATAL: the database system is in recovery mode
DEBUG: proc_exit(0)
DEBUG: shmem_exit(0)
DEBUG: exit(0)
DEBUG: forked new backend, pid=8613 socket=6
DEBUG: reaping dead processes
DEBUG: server process (PID 8613) exited with exit code 0
LOG: all server processes terminated; reinitializing
DEBUG: shmem_exit(0)
DEBUG: invoking IpcMemoryCreate(size=10387456)
LOG: database system was interrupted at 2005-06-15 14:10:18 CEST
LOG: checkpoint record is at 0/35A03C
LOG: redo record is at 0/35A03C; undo record is at 0/0; shutdown FALSE
LOG: next transaction ID: 640; next OID: 24578
LOG: next MultiXactId: 1; next MultiXactOffset: 0
LOG: database system was not properly shut down; automatic recovery in
progress
LOG: redo starts at 0/35A080
LOG: record with zero length at 0/360524
LOG: redo done at 0/3604FC
LOG: database system is ready
LOG: transaction ID wrap limit is 2147484134, limited by database "test"
DEBUG: proc_exit(0)
DEBUG: shmem_exit(0)
DEBUG: exit(0)
DEBUG: reaping dead processes

---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo (AT) postgresql (DOT) org so that your
message can get through to the mailing list cleanly

On Wed, Jun 15, 2005 at 02:29:27PM +0200, Zac wrote:

The crash happens if the Unicode string has the high bit set. For
example, u'\x7f' doesn't cause a crash, but u'\x80' does. Here's
a stack trace from HEAD and Python 2.4.1:

#0 0xfec6967c in PyString_AsString (op=0x0) at Objects/stringobject.c:698
#1 0xfed76a38 in PLy_function_handler (fcinfo=0xffbfde28, proc=0x47c610)
at plpython.c:777
#2 0xfed77df4 in plpython_call_handler (fcinfo=0xffbfde28) at plpython.c:352

Lines 776-77 in plpython.c are:

plrv_so = PyObject_Str(plrv);
plrv_sc = PyString_AsString(plrv_so);

PyObject_Str() is documented to return NULL on failure:

http://www.python.org/doc/2.4.1/api/object.html

Apparently PyString_AsString() isn't expecting a NULL argument, so
the code should probably check the return value of PyObject_Str()
before calling PyString_AsString().

--
Michael Fuhr
http://www.fuhr.org/~mfuhr/

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq