[Solved] Utility.Library/CallHookPkt()/"InitHook" possible?

This forum is for general developer support questions.
Belxjander
Posts: 314
Joined: Mon May 14, 2012 10:26 pm
Location: 日本千葉県松戸市 / Matsudo City, Chiba, Japan
Contact:

[Solved] Utility.Library/CallHookPkt()/"InitHook" possible?

Post by Belxjander »

I'm currently running into a problem where I am needing to embed and extend a Hook structure in the middle of another structure...

Is the new style to "AllocSysObject()" the hook structure and exlusively set Hook structures as 68K unless dynamically allocated otherwise?

I will have to deliberately refactor this change into my existing codebases if this is in fact the case.

Can anyone confirm this?

If this is true...then for what reason would retaining the Hook structure reference be useful in Hook calling for PowerPC systems? (only a backwards compatability for 68K in that case afaik)
Last edited by Belxjander on Wed May 20, 2015 3:45 am, edited 2 times in total.
User avatar
colinw
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 207
Joined: Mon Aug 15, 2011 9:20 am
Location: Brisbane, QLD. Australia.

Re: Utility.Library/CallHookPkt() :: "InitHook" possible?

Post by colinw »

It doesn't currently matter what allocated the hook structure.

IUtility->CallHookPkt() only needs the hook structure h_Entry member initialised, this must point to the code to run,
the type determination is done automatically by checking the memory type pointed to by h_Entry , if the memory
type is MEMF_EXECUTABLE, it is native code, otherwise it's assumed to be 68K code and the emulator is used to run it.

IUtility->CallHookPkt() also does not access any members of the hook structure other than h_Entry, therefore all other
members are available for you to use as you wish, this includes the embedded node structure.
Belxjander
Posts: 314
Joined: Mon May 14, 2012 10:26 pm
Location: 日本千葉県松戸市 / Matsudo City, Chiba, Japan
Contact:

Re: Utility.Library/CallHookPkt() :: "InitHook" possible?

Post by Belxjander »

I am currently attempting to run PPC native code referenced from a Hook structure and seeing 68K Execution crashes within the latest Perception-IME builds.

http://github.com/Belxjander/Perception-IME

the code is in japanese.language and called from perception.library.

is the module cross-calling a detail to care about?
User avatar
colinw
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 207
Joined: Mon Aug 15, 2011 9:20 am
Location: Brisbane, QLD. Australia.

Re: Utility.Library/CallHookPkt() :: "InitHook" possible?

Post by colinw »

If you are experiencing crashes, you need to make sure the hook is being initialised and called properly by
using only the CallHookPkt() function, jumping to the function directly is forbidden.
I presume you have read the autodoc for utility.library CallHookPkt() function ? Here's the important part...

===================================================================================================

CallHookPkt calls the code in h_Entry. This pointer can either be a
pointer to legacy 68k code, or to native code. ExecSG takes care to
place native code into specially marked segments of memory. When h_Entry
points to such a place, CallHookPkt knows that the code is native, and
will directly branch to the code pointer. If it doesn't, then the code
is assumed to be non-native (i.e. legacy 68k code), and the emulator
is invoked at this code address.

This form of transparency comes at a price: h_Entry must not be jumped
to directly, as calling 68k legacy code from native code will result in
a crash, and calling native code from 68k legacy code will result in
native code being interpreted as 68k code, with completely unpredictable results.
Belxjander
Posts: 314
Joined: Mon May 14, 2012 10:26 pm
Location: 日本千葉県松戸市 / Matsudo City, Chiba, Japan
Contact:

Re: Utility.Library/CallHookPkt() :: "InitHook" possible?

Post by Belxjander »

Am I using IUtility->CallHookPkt()... yes,

Am I using IExec->AllocSysObjectTags ().... yes

Is the Hook code native PowerPC... yes

The crash results on CallHookPkt() executing... attempting to execute 68K code and not PowerPC? apparently.

I'm not 100% sure of the issue as the 440 I am using deadlocks at the time of the crash.

I'm willing to send a minimal archive of whats needed to reproduce the crash...
or feel free to use sgit for cloning and building the 2015/May/18th code and build it yourself.
User avatar
colinw
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 207
Joined: Mon Aug 15, 2011 9:20 am
Location: Brisbane, QLD. Australia.

Re: Utility.Library/CallHookPkt() :: "InitHook" possible?

Post by colinw »

Belxjander wrote: Am I using IUtility->CallHookPkt()... yes,
Am I using IExec->AllocSysObjectTags ().... yes
Is the Hook code native PowerPC... yes
The crash results on CallHookPkt() executing... attempting to execute 68K code and not PowerPC? apparently.
If the hook is pointing to native PPC code, then it must be in MEMF_EXECUTABLE type memory for it to be identified as such.
This happens automatically when loaded by IDOS->LoadSeg() or the elf.library functions. (which DOS uses internally).
However, if the code is being interpreted as 68K, then the only thing I can think of right now, is that somehow
it was loaded by "some other means" and is residing in the wrong memory type.

You could always see what IExec->IsNative() says about it, just to confirm my guess; eg:

==============================================================================================
if( IExec->IsNative(hook->h_Entry) )
{
IExec->DebugPrintF("h_Entry points to native PPC code. [addr=0x%08lx]\n",hook->h_Entry);
}
else
{
IExec->DebugPrintF("h_Entry will be interpreted as 68K code. [addr=0x%08lx]\n",hook->h_Entry);
}
==============================================================================================
(See also; IExec->TypeOfMem() for a general memory type testing function.)

If the above test shows that the hook is indeed pointing to native PPC code, then it is also possible that
the native PPC code is actually executing perfectly, but it somehow call a function pointer that points
to 68K code, which makes it look like the PPC code was mis-identified as 68K.

A crashlog or two posted here may also help.
Belxjander
Posts: 314
Joined: Mon May 14, 2012 10:26 pm
Location: 日本千葉県松戸市 / Matsudo City, Chiba, Japan
Contact:

Re: Utility.Library/CallHookPkt() :: "InitHook" possible?

Post by Belxjander »

I will check this at the first opportunity.

Just a note...

Segments are loaded from LIBS:Perception.Library and LOCALE:Languages/japanese.language
the loading command is Iprefs which is part of the OS distribution.

IExec->OpenLibrary() always uses IDOS->LoadSeg() afaik
User avatar
broadblues
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 600
Joined: Sat Jun 18, 2011 2:40 am
Location: Portsmouth, UK
Contact:

Re: Utility.Library/CallHookPkt() :: "InitHook" possible?

Post by broadblues »

Excuse the obviosu comment but sometimes the obvious is overlooked, but when code jumps intu no where, that often means insufficient stack...
User avatar
colinw
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 207
Joined: Mon Aug 15, 2011 9:20 am
Location: Brisbane, QLD. Australia.

Re: Utility.Library/CallHookPkt() :: "InitHook" possible?

Post by colinw »

broadblues wrote:Excuse the obvious comment but sometimes the obvious is overlooked, but when code jumps intu no where,
that often means insufficient stack...
It's more likely an uninitialised pointer.

If the stack was low, the last push would have likely damaged one of the cookies at stackbase and the crashlog would
indicate this by "Stack cookies damaged" in grimreaper. However, reading from past the end of the stack would generally
only cause a page exception DSI, not a code crash (ISI). I believe the last time I looked, exec places a 4K guard page
below stackbase, just to make sure it crashes.

Of course the obvious thing to do is use "Ranger" see ->[DOS]->[Processes] and actually check to see what percentage
of the stack space remains unused, then just make sure it stays <75% by adding a stack cookie to your program, if it's short.
See; IDOS->FindSegmentStackSize() for details about adding a stack cookie to your program, if you need it to have
more than the default 16K. It's far safer than just guessing.

While you are developing software, always make sure you set the MUNGE keyword in the kernel commandline,
it will show up uninitialised pointers and free memory accesses straight away.
Belxjander
Posts: 314
Joined: Mon May 14, 2012 10:26 pm
Location: 日本千葉県松戸市 / Matsudo City, Chiba, Japan
Contact:

Re: Utility.Library/CallHookPkt() :: "InitHook" possible?

Post by Belxjander »

Stack Allocated when launching the process is a healthy 128KBytes, current average usage is within 32KBytes.

I over-allocated to track stack usage during development with the idea to trim it back later.

I may leave it as is due to potential recursive search usage within the plugins ( Japanese at least looks to need roughly 24-30KBytes of it's own data structures for temporary data and át least 1-2MB of dictionary cache, possibly up to 16MB.

I'm still trying to work out what search options and data layout works best for Japanese.

Currently perception.library loads and launches a master thread which I then register resource usage from.
the perception.library and Perception-IME process work as two contexts with isolation of IME plugins handling InputTagItem structured data (modified equivalent size to TagItem structs) so that crashing the IME plugin processing doesn't crash input.device.(the Perception-IME process is processing a FIFO of InputTagItem by calling language plugins.

I'm trying to do this "system safe" and using a "LanguageContext" structure for input/process and output state to be retained across plugin calls.

I am noticing there are human readable values as part of the sashimi output before a full lockup from crashing.

any recommendable instructions for wiring a serial cable to an RPi for logging debugging there?


EDIT: Updated the codebase, found that I was iterating an extra node which was pointed somewhere strange.
I have now included a limitation for IsNative() prior to calling the LanguageHook entirely without further crashing.

My next steps will be to rebuild the refactored Japanese.Language Hook codebase and begin the task of internal protocol refactoring to account for an extended set of capabilities.
Post Reply