Manually loading shared object

This forum is for general developer support questions.
softwarefailure
Posts: 112
Joined: Fri Feb 14, 2014 10:29 pm

Manually loading shared object

Post by softwarefailure »

How can I manually open shared objects and import their symbols at runtime? I've tried the following:

Code: Select all

#include <stdio.h>

#include <exec/exec.h>
#include <exec/types.h>

#include <dos/dos.h>
#include <dos/dostags.h>
#include <libraries/elf.h>

#include <proto/dos.h>
#include <proto/exec.h>
#include <proto/elf.h>

#define SDL_INIT_TIMER          0x00000001
#define SDL_INIT_AUDIO          0x00000010
#define SDL_INIT_VIDEO          0x00000020  /**< SDL_INIT_VIDEO implies SDL_INIT_EVENTS */
#define SDL_INIT_JOYSTICK       0x00000200  /**< SDL_INIT_JOYSTICK implies SDL_INIT_EVENTS */
#define SDL_INIT_HAPTIC         0x00001000
#define SDL_INIT_GAMECONTROLLER 0x00002000  /**< SDL_INIT_GAMECONTROLLER implies SDL_INIT_JOYSTICK */
#define SDL_INIT_EVENTS         0x00004000

static int (*m_SDL_Init)(ULONG flags);

int main(int argc, char *argv[])
{
	struct Process *self;
	BPTR seglist;
	APTR result;
	APTR sdllib;
	Elf32_Handle elfHandle;

	self = (struct Process *) FindTask(0);
	seglist = GetProcSegList(self, GPSLF_CLI|GPSLF_SEG);

	GetSegListInfoTags(seglist, GSLI_ElfHandle, &elfHandle, TAG_DONE);

	elfHandle = OpenElfTags(OET_ElfHandle, elfHandle, TAG_DONE);
	sdllib = DLOpen(elfHandle, "libSDL2.so", 0);

	DLSym(elfHandle, sdllib, "SDL_Init", &result);
	m_SDL_Init = result;

	printf("HMM: %d\n", m_SDL_Init(SDL_INIT_VIDEO|SDL_INIT_TIMER|SDL_INIT_JOYSTICK|SDL_INIT_GAMECONTROLLER|SDL_INIT_AUDIO));

	DLClose(elfHandle, sdllib);
	CloseElfTags(elfHandle, CET_ReClose, TRUE, TAG_DONE);

	return 0;
} 
However, it does not work. Calling m_SDL_Init() results in the following error message from ELF Library: "Unable to resolve symbol: pthread_key_create". And after that: "Failed to resolve symbol at runtime. Process ... has been suspended".

Why does this not work and how can I fix it?

The ultimate goal is to be able to use libSDL2.so from a project that is linked without compiler constructor/destructor code, i.e. with the -nostartfiles option. The code above is just a test. The code above has been compiled using:

Code: Select all

gcc -D__USE_INLINE__ test.c -lauto -lpthread
Tested with r164 of libSDL2.so from here: https://sourceforge.net/projects/sdl2-a ... rce=navbar
User avatar
broadblues
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 600
Joined: Sat Jun 18, 2011 2:40 am
Location: Portsmouth, UK
Contact:

Re: Manually loading shared object

Post by broadblues »

softwarefailure wrote:How can I manually open shared objects and import their symbols at runtime? I've tried the following:

Code: Select all

#include <stdio.h>

#include <exec/exec.h>
#include <exec/types.h>

#include <dos/dos.h>
#include <dos/dostags.h>
#include <libraries/elf.h>

#include <proto/dos.h>
#include <proto/exec.h>
#include <proto/elf.h>

#define SDL_INIT_TIMER          0x00000001
#define SDL_INIT_AUDIO          0x00000010
#define SDL_INIT_VIDEO          0x00000020  /**< SDL_INIT_VIDEO implies SDL_INIT_EVENTS */
#define SDL_INIT_JOYSTICK       0x00000200  /**< SDL_INIT_JOYSTICK implies SDL_INIT_EVENTS */
#define SDL_INIT_HAPTIC         0x00001000
#define SDL_INIT_GAMECONTROLLER 0x00002000  /**< SDL_INIT_GAMECONTROLLER implies SDL_INIT_JOYSTICK */
#define SDL_INIT_EVENTS         0x00004000

static int (*m_SDL_Init)(ULONG flags);

int main(int argc, char *argv[])
{
	struct Process *self;
	BPTR seglist;
	APTR result;
	APTR sdllib;
	Elf32_Handle elfHandle;

	self = (struct Process *) FindTask(0);
	seglist = GetProcSegList(self, GPSLF_CLI|GPSLF_SEG);

	GetSegListInfoTags(seglist, GSLI_ElfHandle, &elfHandle, TAG_DONE);

	elfHandle = OpenElfTags(OET_ElfHandle, elfHandle, TAG_DONE);
	sdllib = DLOpen(elfHandle, "libSDL2.so", 0);

	DLSym(elfHandle, sdllib, "SDL_Init", &result);
	m_SDL_Init = result;

	printf("HMM: %d\n", m_SDL_Init(SDL_INIT_VIDEO|SDL_INIT_TIMER|SDL_INIT_JOYSTICK|SDL_INIT_GAMECONTROLLER|SDL_INIT_AUDIO));

	DLClose(elfHandle, sdllib);
	CloseElfTags(elfHandle, CET_ReClose, TRUE, TAG_DONE);

	return 0;
} 
However, it does not work. Calling m_SDL_Init() results in the following error message from ELF Library: "Unable to resolve symbol: pthread_key_create". And after that: "Failed to resolve symbol at runtime. Process ... has been suspended".

Why does this not work and how can I fix it?

The ultimate goal is to be able to use libSDL2.so from a project that is linked without compiler constructor/destructor code, i.e. with the -nostartfiles option. The code above is just a test. The code above has been compiled using:

Code: Select all

gcc -D__USE_INLINE__ test.c -lauto -lpthread
Tested with r164 of libSDL2.so from here: https://sourceforge.net/projects/sdl2-a ... rce=navbar
Your application needs to be dynamically linked at the very least. Manual low level elf.library calls are not going to be easy to use maybe try libdl? Perl uses libdl to load it's modules (as probably does python) if your surnames not Frieden you probably don;t want to mess with elf.library directly

See what just changing to this does....

gcc -D__USE_INLINE__ -use-dynld test.c -lauto -lpthread
softwarefailure
Posts: 112
Joined: Fri Feb 14, 2014 10:29 pm

Re: Manually loading shared object

Post by softwarefailure »

broadblues wrote:Manual low level elf.library calls are not going to be easy to use maybe try libdl? Perl uses libdl to load it's modules (as probably does python
I've tried to use dlopen(), dlsym() and dlclose() directly but it doesn't work at linker stage. The linker complains about being unable to resolve the dlopen(), dlsym() and dlclose() symbols which isn't surprising because there doesn't seem to be any linker library in the newlib SDK target that has those symbols. clib2 seems to have them but not newlib.
See what just changing to this does....
gcc -D__USE_INLINE__ -use-dynld test.c -lauto -lpthread
When using this the error message goes away, but instead the program instantly crashes when calling m_SDL_Init(), so it doesn't work either.
if your surnames not Frieden you probably don;t want to mess with elf.library directly
Well, I'll do whatever it takes to get this working. Manually opening the shared object and importing the symbols seemed to be the most straightforward way but apparently there is so more magic to it, so please someone advise what I can do to get this working.
User avatar
broadblues
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 600
Joined: Sat Jun 18, 2011 2:40 am
Location: Portsmouth, UK
Contact:

Re: Manually loading shared object

Post by broadblues »

softwarefailure wrote:
broadblues wrote:Manual low level elf.library calls are not going to be easy to use maybe try libdl? Perl uses libdl to load it's modules (as probably does python
I've tried to use dlopen(), dlsym() and dlclose() directly but it doesn't work at linker stage. The linker complains about being unable to resolve the dlopen(), dlsym() and dlclose() symbols which isn't surprising because there doesn't seem to be any linker library in the newlib SDK target that has those symbols. clib2 seems to have them but not newlib.
There is no libdl.a just a libdl.so You have to use dynamic linking. there should be alink in your SDK to the copy in SOBJS:
See what just changing to this does....
gcc -D__USE_INLINE__ -use-dynld test.c -lauto -lpthread
When using this the error message goes away, but instead the program instantly crashes when calling m_SDL_Init(), so it doesn't work either.
Perhaps the shared objects constructors need calling?

Most of them have symbols like

__shlib_call_constructors etc

don't know what the prototype for that would be.


I'm thinking that using libdl will call the constructors for you, but using elf.library direct you would need to call them youself, which involves knowing how to call them properly (or guessing I suppose).

My much earlier perl port used elf library calls IIRC but the modules weren't true shared objects, just custom binaries, when I switched to newlib I started using proper shared objects for the modules and switched to newlibs libdl.so (in part because there was existing perl code to easily adapt to the job).
softwarefailure
Posts: 112
Joined: Fri Feb 14, 2014 10:29 pm

Re: Manually loading shared object

Post by softwarefailure »

Ok, I've now tested the following code:

Code: Select all

#include <stdio.h>

#include <exec/exec.h>
#include <exec/types.h>

#include <dos/dos.h>
#include <dos/dostags.h>
#include <libraries/elf.h>

#include <proto/dos.h>
#include <proto/exec.h>
#include <proto/elf.h>

#include <dlfcn.h>

#define SDL_INIT_TIMER          0x00000001
#define SDL_INIT_AUDIO          0x00000010
#define SDL_INIT_VIDEO          0x00000020  /**< SDL_INIT_VIDEO implies SDL_INIT_EVENTS */
#define SDL_INIT_JOYSTICK       0x00000200  /**< SDL_INIT_JOYSTICK implies SDL_INIT_EVENTS */
#define SDL_INIT_HAPTIC         0x00001000
#define SDL_INIT_GAMECONTROLLER 0x00002000  /**< SDL_INIT_GAMECONTROLLER implies SDL_INIT_JOYSTICK */
#define SDL_INIT_EVENTS         0x00004000

static int (*m_SDL_Init)(ULONG flags);

int main(int argc, char *argv[])
{
	void *plug;
	
	plug = dlopen("libSDL2.so", RTLD_LAZY);

	m_SDL_Init = dlsym(plug, "SDL_Init");

	printf("CHECK: %p\n", m_SDL_Init);

	printf("HMM: %d\n", m_SDL_Init(SDL_INIT_VIDEO|SDL_INIT_TIMER|SDL_INIT_JOYSTICK|SDL_INIT_GAMECONTROLLER|SDL_INIT_AUDIO));

	dlclose(plug);

	return 0;
}
Compiled with:

Code: Select all

gcc -use-dynld test.c -lpthread -ldl
But it crashes when calling m_SDL_Init() in the very same fashion as my previous version which used elf.library directly.

The following code, however, works:

Code: Select all

#include <stdio.h>

#include <exec/exec.h>
#include <exec/types.h>

#include <dos/dos.h>
#include <dos/dostags.h>
#include <libraries/elf.h>

#include <proto/dos.h>
#include <proto/exec.h>
#include <proto/elf.h>

#include <SDL.h>

#define SDL_INIT_TIMER          0x00000001
#define SDL_INIT_AUDIO          0x00000010
#define SDL_INIT_VIDEO          0x00000020  /**< SDL_INIT_VIDEO implies SDL_INIT_EVENTS */
#define SDL_INIT_JOYSTICK       0x00000200  /**< SDL_INIT_JOYSTICK implies SDL_INIT_EVENTS */
#define SDL_INIT_HAPTIC         0x00001000
#define SDL_INIT_GAMECONTROLLER 0x00002000  /**< SDL_INIT_GAMECONTROLLER implies SDL_INIT_JOYSTICK */
#define SDL_INIT_EVENTS         0x00004000

int main(int argc, char *argv[])
{
	printf("HMM: %d\n", SDL_Init(SDL_INIT_VIDEO|SDL_INIT_TIMER|SDL_INIT_JOYSTICK|SDL_INIT_GAMECONTROLLER|SDL_INIT_AUDIO));

	return 0;
} 
Compiled with:

Code: Select all

gcc -use-dynld test.c -lSDL2 -lpthread
So I think what we need to find out is how the second version actually initializes libSDL2.so and then imitate this behaviour in the first code... has really nobody ever tried this before? I don't think it is such an exotic thing to do to try to manually open a shared object instead of having the compiler constructor do this...
User avatar
Thomas Frieden
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 147
Joined: Fri Dec 10, 2010 3:21 pm

Re: Manually loading shared object

Post by Thomas Frieden »

Check if libSDL.so is correctly linked against libpthread.so.

readelf -d libSDL.so

It should output something like this:

Code: Select all

Dynamic section at offset 0x7a92c contains 20 entries:
  Tag        Type                         Name/Value
 0x00000001 (NEEDED)                     Shared library: [libpthread.so]
 0x00000001 (NEEDED)                     Shared library: [libstdc++.so]
 0x00000001 (NEEDED)                     Shared library: [libgcc.so]
 0x00000001 (NEEDED)                     Shared library: [libc.so]
User avatar
Thomas Frieden
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 147
Joined: Fri Dec 10, 2010 3:21 pm

Re: Manually loading shared object

Post by Thomas Frieden »

Just noticed you are using a publically available libSDL.so... checked myself:

Code: Select all

$ ppc-amigaos-readelf  -d libSDL2-2.0.so

Dynamic section at offset 0x123420 contains 18 entries:
  Tag        Type                         Name/Value
 0x00000001 (NEEDED)                     Shared library: [libgcc.so]
 0x00000001 (NEEDED)                     Shared library: [libc.so]
 0x0000000e (SONAME)                     Library soname: [libSDL2-2.0.so]
 0x00000004 (HASH)                       0x100000b4
 0x00000005 (STRTAB)                     0x100056b0
 0x00000006 (SYMTAB)                     0x10001870
 0x0000000a (STRSZ)                      17854 (bytes)
 0x0000000b (SYMENT)                     16 (bytes)
 0x00000003 (PLTGOT)                     0x10106000
 0x00000002 (PLTRELSZ)                   7236 (bytes)
 0x00000014 (PLTREL)                     RELA
 0x00000017 (JMPREL)                     0x1001c27c
 0x00000007 (RELA)                       0x10009c70
 0x00000008 (RELASZ)                     82512 (bytes)
 0x00000009 (RELAENT)                    12 (bytes)
 0x6000000e (AMIGAOS_DYNVERSION)         0x2
 0x6ffffff9 (RELACOUNT)                  3877
So, no.

Shared objects need to be linked against the libraries they use. Linking the main program against a library does not resolve correctly if in this case libSDL is linked before libpthread
softwarefailure
Posts: 112
Joined: Fri Feb 14, 2014 10:29 pm

Re: Manually loading shared object

Post by softwarefailure »

Thanks, so are you saying that the problem is in libSDL.so? Should I report this to the OS4 SDL maintainers so that they link libSDL.so correctly against libpthread?
User avatar
Thomas Frieden
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 147
Joined: Fri Dec 10, 2010 3:21 pm

Re: Manually loading shared object

Post by Thomas Frieden »

softwarefailure wrote:Thanks, so are you saying that the problem is in libSDL.so? Should I report this to the OS4 SDL maintainers so that they link libSDL.so correctly against libpthread?
Yes. It has to be linked against all respective libraries
capehill
Posts: 24
Joined: Sun Jun 18, 2017 1:07 pm

Re: Manually loading shared object

Post by capehill »

Hi all,

I haven't yet figured out how to pass -lpthread to libtool (for configure/make build) but I'm able to build the r178 using cmake (+gmake), and that produces a shared object linked with pthreads.

But unfortunately it is not compatible with the old binaries that were linked with older libSDL2.so and this is bad. Crash is somewhere in pthread/kernel. So I'm wondering is this caused when both the .so and the binary are dynamically linked with pthreads or is there something else.. I will post a stack trace later when I get back to my Amiga system. EDIT: this issue seems to be solved now and can be ignored.
Last edited by capehill on Fri Jun 30, 2017 4:44 pm, edited 1 time in total.
Post Reply