Still, Python seems to stalk me. (I trusssts it not)
(Note that this of course is the abbreviated tale. Otherwise, this post would have been this B C-u 10000 I G.)
I was working through Dan Luu's malloc tutorial (more on that in the long promised malloc post!) and after doing the 'export LD_PRELOAD=...' trick and making the world run on my freshly typed toy malloc.so, gdb and clang bugged out thus on Ubuntu 14.04:
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: Py_Initialize: can't import _frozen_importlib
IndexError: list assignment index out of range
Aborted (core dumped)
Lucky for me, after a while, out of the blue, the xcfe error reporter pops up and has this useful info:
Stack trace: /usr/lib/i386-linux-gnu/libpython3.4m.so.1.0
is the lib that appears to cause the problem. A quick ldd confirms that it's indeed involved.
But of course, python itself works. But hang on, (after some deliberation...) python3 does not. Aha!
strace python3 indeed does spit venom:
fstat(0, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 72), ...}) = 0
brk(0xb03f39) = 0xb03f39
readlink("/usr/bin/python3", "python3.4", 4096) = 9
readlink("/usr/bin/python3.4", 0x7fffff316970, 4096) = -1 EINVAL (Invalid argument)
open("/usr/bin/pyvenv.cfg", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/pyvenv.cfg", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/usr/bin/Modules/Setup", 0x7fffff3178f0) = -1 ENOENT (No such file or directory)
stat("/usr/bin/lib/python3./os.py", 0x7fffff3178f0) = -1 ENOENT (No such file or directory)
stat("/usr/bin/lib/python3./os.pyc", 0x7fffff3178f0) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python3./os.py", 0x7fffff3178f0) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python3./os.pyc", 0x7fffff3178f0) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python3./os.py", 0x7fffff3178f0) = -1 ENOENT (No such file or directory)
stat("/usr/lib/python3./os.pyc", 0x7fffff3178f0) = -1 ENOENT (No such file or directory)
write(2, "Could not find platform independ"..., 55Could not find platform independent libraries <prefix>
) = 55
But here the snake bites it's tail, since I now first have to debug the debugging process before any debugging can take place --- Dan kindly sent me a cool wrapper trick, so the LD_PRELOAD happens in the gdb, rather than outside of it, and to gdb. As an aside, Dan is not seeing the bug I'm seeing, but a friend of mine whom I asked to check on his setup, promptly has his prompt killed by this bug, so this is a distro specific adventure, with varying outcomes.
So, I grab the python3.4 source and the Ubuntu patch (just to be throrough) and set it all up.
The next puzzle was how to get Emacs's M-x gdb working with this. After some advanced handwaving, the following incantation coalesces:
1. M-x gdb
2. Run gdb (like this): gdb --annotate=3 -i=mi /home/g/python_source/python3.4-3.4.0/python
3. (gdb) set exec-wrapper /home/g/c-stuff/lpi/tlpi-dist/exercises/wrapper
4. break main
Fiddling about a bit, gets me this call stack:
0 in __GI_raise of ../nptl/sysdeps/unix/sysv/linux/raise.c:32
1 in __GI_abort of abort.c:89
2 in Py_FatalError of Python/pythonrun.c:2628
3 in import_init of Python/pythonrun.c:280
4 in _Py_InitializeEx_Private of Python/pythonrun.c:445
5 in Py_InitializeEx of Python/pythonrun.c:489
6 in Py_Initialize of Python/pythonrun.c:490
7 in Py_Main of Modules/main.c:654
8 in main of ./Modules/python.c:69
And chasing the snake down the rabbit hole a bit further gives me this in raise.c on line 32:
pid_t selftid = THREAD_GETMEM (pd, tid);
which in turn is defined in tls.h as
# define THREAD_GETMEM(descr, member) descr->member
At which point I've definitely arrived in Wonderland and I have to just guess that pi*thumb the problem is that the toy malloc isn't thread safe.
The toy malloc is first used here by python:
0 in malloc of malloc.c:67
=>in main of ./Modules/python.c:30.
and the pertaining line of code is this:
argv_copy = (wchar_t **)PyMem_RawMalloc(sizeof(wchar_t*) * (argc+1));
which is defined as
void *
PyMem_RawMalloc(size_t size)
{
/*
* Limit ourselves to PY_SSIZE_T_MAX bytes to prevent security holes.
* Most python internals blindly use a signed Py_ssize_t to track
* things without checking for overflows or negatives.
* As size_t is unsigned, checking for size < 0 is not required.
*/
if (size > (size_t)PY_SSIZE_T_MAX)
return NULL;
return _PyMem_Raw.malloc(_PyMem_Raw.ctx, size);
}
rooting around in the python3.4-3.4.0/Objects/obmalloc.c file I can see malloc being used in a number of places, but the shape is somewhat esoteric and I'm not quite sure about exactly what happens there(there be dragons!), but at some point, the toy malloc is employed.
Actually, there is one kind of snake I do like. It's one of my favourite poems by D.H. Lawrence:
Snake
A snake came to my water-trough
On a hot, hot day, and I in pyjamas for the heat,
To drink there.
In the deep, strange-scented shade of the great dark carob-tree
I came down the steps with my pitcher
And must wait, must stand and wait, for there he was at the trough before
me.
He reached down from a fissure in the earth-wall in the gloom
And trailed his yellow-brown slackness soft-bellied down, over the edge of
the stone trough
And rested his throat upon the stone bottom,
And where the water had dripped from the tap, in a small clearness,
He sipped with his straight mouth,
Softly drank through his straight gums, into his slack long body,
Silently.
Someone was before me at my water-trough,
And I, like a second comer, waiting.
He lifted his head from his drinking, as cattle do,
And looked at me vaguely, as drinking cattle do,
And flickered his two-forked tongue from his lips, and mused a moment,
And stooped and drank a little more,
Being earth-brown, earth-golden from the burning bowels of the earth
On the day of Sicilian July, with Etna smoking.
The voice of my education said to me
He must be killed,
For in Sicily the black, black snakes are innocent, the gold are venomous.
And voices in me said, If you were a man
You would take a stick and break him now, and finish him off.
But must I confess how I liked him,
How glad I was he had come like a guest in quiet, to drink at my water-trough
And depart peaceful, pacified, and thankless,
Into the burning bowels of this earth?
Was it cowardice, that I dared not kill him?
Was it perversity, that I longed to talk to him?
Was it humility, to feel so honoured?
I felt so honoured.
And yet those voices:
If you were not afraid, you would kill him!
And truly I was afraid, I was most afraid,
But even so, honoured still more
That he should seek my hospitality
From out the dark door of the secret earth.
He drank enough
And lifted his head, dreamily, as one who has drunken,
And flickered his tongue like a forked night on the air, so black,
Seeming to lick his lips,
And looked around like a god, unseeing, into the air,
And slowly turned his head,
And slowly, very slowly, as if thrice adream,
Proceeded to draw his slow length curving round
And climb again the broken bank of my wall-face.
And as he put his head into that dreadful hole,
And as he slowly drew up, snake-easing his shoulders, and entered farther,
A sort of horror, a sort of protest against his withdrawing into that horrid black hole,
Deliberately going into the blackness, and slowly drawing himself after,
Overcame me now his back was turned.
I looked round, I put down my pitcher,
I picked up a clumsy log
And threw it at the water-trough with a clatter.
I think it did not hit him,
But suddenly that part of him that was left behind convulsed in undignified haste.
Writhed like lightning, and was gone
Into the black hole, the earth-lipped fissure in the wall-front,
At which, in the intense still noon, I stared with fascination.
And immediately I regretted it.
I thought how paltry, how vulgar, what a mean act!
I despised myself and the voices of my accursed human education.
And I thought of the albatross
And I wished he would come back, my snake.
For he seemed to me again like a king,
Like a king in exile, uncrowned in the underworld,
Now due to be crowned again.
And so, I missed my chance with one of the lords
Of life.
And I have something to expiate:
A pettiness.
Taormina, 1923
