SDK:C/make very slow with OS4.1FE (SOLVED)

Have a question about our Software Developer Kit? Ask them here.
zzd10h
Posts: 546
Joined: Sun Sep 16, 2012 5:40 am
Location: France

Re: SDK:C/make very slow with OS4.1FE

Post by zzd10h »

@xenic

As I have confirmed the slowdown by compiling SMtube on FE compared to 4.1.6, here :

http://forum.hyperion-entertainment.biz ... 845#p31852

could you send me your fixed make command (and gmake too if you have it) please ?

I will see if SMTube is faster to compile on FE.

Thank you
guillaume at boesel . fr
http://apps.amistore.net/zTools
X1000 - AmigaOS 4.1.6 / 4.1 FE
User avatar
tonyw
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 1479
Joined: Wed Mar 09, 2011 1:36 pm
Location: Sydney, Australia

Re: SDK:C/make very slow with OS4.1FE

Post by tonyw »

whereas there doesn't seem to be any way to stop make 53.1 once it starts a compile.
What? You can't stop a make with Ctrl/C? I can and do, although I'm not running multiple levels with sub-makefiles.
cheers
tony
xenic
Posts: 1185
Joined: Sun Jun 19, 2011 12:06 am

Re: SDK:C/make very slow with OS4.1FE

Post by xenic »

tonyw wrote:
whereas there doesn't seem to be any way to stop make 53.1 once it starts a compile.
What? You can't stop a make with Ctrl/C? I can and do, although I'm not running multiple levels with sub-makefiles.
You're absolutely right. Ctrl-C works with make 53.1 at the top level. When you have a top level makefile calling sub-makes, Ctrl-C doesn't always work. However, if you enter Ctrl-C in a sub-make when make has compiled one file and hasn't yet started another, it will break. That can be difficult to do though.
AmigaOne X1000 with 2GB memory - OS4.1 FE
xenic
Posts: 1185
Joined: Sun Jun 19, 2011 12:06 am

Re: SDK:C/make very slow with OS4.1FE

Post by xenic »

I copied my OS4.1FE to an FFS partition and "make" operates at normal speed When I boot from that partition. I followed advice and formatted my OS4.1FE partition as SFS0. I need to investigate further but it seems like SFS0 is the cause of my problem with the make command.
AmigaOne X1000 with 2GB memory - OS4.1 FE
xenic
Posts: 1185
Joined: Sun Jun 19, 2011 12:06 am

Re: SDK:C/make very slow with OS4.1FE

Post by xenic »

Further testing shows that the make (v53.1) command operates at the same speed when I boot from an OS4.1FE FFS partition as it does when I boot from an OS4.1u6 SFS2 partition. It makes no sense to me but here are my timings for compiling the Dopus5 program after cold booting (reset) from different partitions:

Booting with OS4.1u6 SFS2 partition: 225.2183 seconds
Booting with OS4.1FE FFS partition: 217.1784 seconds
Booting with OS4.1FE SFS0 partition: 541.3016 seconds
Booting with OS4.1FE SFS2 partition: 546.6680 seconds

Something is wrong but I can't imagine what it is.
AmigaOne X1000 with 2GB memory - OS4.1 FE
xenic
Posts: 1185
Joined: Sun Jun 19, 2011 12:06 am

Re: SDK:C/make very slow with OS4.1FE

Post by xenic »

I did some more testing and tracked the problem to Python. If I edit the startup-sequence on an SFS2 boot partition, change the PYTHON: assignment to the Python directory on an FFS partition and cold boot from that SFS2 partition then "make" works at normal speed (as fast as it did with Update6).

I could be wrong, but because Snoopy output of a compile shows the make command starting a process called make.spawn, I suspect that the make command may be spawning processes using Python. Apparently, something was changed in OS4.1FE Python that is causing it (or programs using it) to be slower when Python is located on an SFS or SFS2 partition. Maybe someone needs to take a look at what was changed in Python from Update6 to FE.

Nobody seems very interrested in this problem so I guess it will stay as it is.
AmigaOne X1000 with 2GB memory - OS4.1 FE
User avatar
broadblues
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 600
Joined: Sat Jun 18, 2011 2:40 am
Location: Portsmouth, UK
Contact:

Re: SDK:C/make very slow with OS4.1FE

Post by broadblues »

@Xenic

Just because no one has come up with a solution or narrowed down the problem doesn't mean no one is ineterested. Bear in mind we are not fulltime paid testers, and we have other projects which we are working on too.

Now on to your python theory, this is a strange one, I can't see how python could make any difference at all. It's been updated yes, but not in any unusually way. Make certainly doesn not use it to spawn processes, it worries me that you can come up with such a theory from snoopy log which doesn't mention python at all!

The most recent make (53.1) sets up a much more complex environment for executing subshells (be they AmigaOS shell or abc-shell) which includes a wrapper process for setting environment variables, this is why you see the make.spawn and make.system processes. It's essential for certain types of make file, though not every makefile requires it.

Anyway I tried removing python from the path, and removing the assign completely. The test makefile composed of mini scripts I have been using above still runs slowly at normal priority.

I also tried it on my X1000 and again it runs slowly.

But my real make file for my projects SketchBlock / <top secret new project> / PPaint all run fast with no obvious delays. Although the SB makefile is very complex, none of them use recursion, so that may be some kind of issue.

Makefile design clearly plays a factor.

I noticed one interesting factor so far. If I make DELETE resident then the test makefile with the DEL script runs much faster. 1.5 seconds instead of 9 seconds on my SAM. That made me think it could be path related in some way but setting CC to "gcc" rather than "SDK:gcc/C/gcc" made no difference, nor did calling C:DELETE

Unfortunatly snoopy doesn't have a time stamp column which makes it hard to see whther the DEL script is running slowly or the make file is . ie is the pause occuring at the start of a subshell or between commands in a multicommand rule?
User avatar
broadblues
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 600
Joined: Sat Jun 18, 2011 2:40 am
Location: Portsmouth, UK
Contact:

Re: SDK:C/make very slow with OS4.1FE

Post by broadblues »

tonyw wrote:
whereas there doesn't seem to be any way to stop make 53.1 once it starts a compile.
What? You can't stop a make with Ctrl/C? I can and do, although I'm not running multiple levels with sub-makefiles.
The issue is more likely with abc-shell than make, abc-shell does not play well with CTRL-C at all, sometimes it's passed onto the sub task / command being run sometimes not.
User avatar
tonyw
AmigaOS Core Developer
AmigaOS Core Developer
Posts: 1479
Joined: Wed Mar 09, 2011 1:36 pm
Location: Sydney, Australia

Re: SDK:C/make very slow with OS4.1FE

Post by tonyw »

That may be it, then. I'm only using OS4-native utilities, not abc-shell, for instance.

Python is only a distraction in my environment, although it may be used somehow in xenic's environment. There are differences between the 4.1.6 version and the FE version (using dirdiff).
cheers
tony
xenic
Posts: 1185
Joined: Sun Jun 19, 2011 12:06 am

Re: SDK:C/make very slow with OS4.1FE

Post by xenic »

broadblues wrote:@Xenic
Just because no one has come up with a solution or narrowed down the problem doesn't mean no one is ineterested. Bear in mind we are not fulltime paid testers, and we have other projects which we are working on too.[/qoute]
It's just discouraging when a problem is difficult to reproduce or explain.
Now on to your python theory, this is a strange one, I can't see how python could make any difference at all. It's been updated yes, but not in any unusually way. Make certainly doesn not use it to spawn processes, it worries me that you can come up with such a theory from snoopy log which doesn't mention python at all!
I was switching boot partitions so much that I must have gotten confused. I retested today and Python doesn't have anything to do with the make compile times.
The most recent make (53.1) sets up a much more complex environment for executing subshells (be they AmigaOS shell or abc-shell) which includes a wrapper process for setting environment variables, this is why you see the make.spawn and make.system processes. It's essential for certain types of make file, though not every makefile requires it.
I was wrong about the Python/spawn connection. After more testing it seems unrelated.
Anyway I tried removing python from the path, and removing the assign completely. The test makefile composed of mini scripts I have been using above still runs slowly at normal priority.
As noted above, I as wrong about any python connection.
I also tried it on my X1000 and again it runs slowly.

But my real make file for my projects SketchBlock / <top secret new project> / PPaint all run fast with no obvious delays. Although the SB makefile is very complex, none of them use recursion, so that may be some kind of issue.

Makefile design clearly plays a factor.

I noticed one interesting factor so far. If I make DELETE resident then the test makefile with the DEL script runs much faster. 1.5 seconds instead of 9 seconds on my SAM. That made me think it could be path related in some way but setting CC to "gcc" rather than "SDK:gcc/C/gcc" made no difference, nor did calling C:DELETE

Unfortunatly snoopy doesn't have a time stamp column which makes it hard to see whther the DEL script is running slowly or the make file is . ie is the pause occuring at the start of a subshell or between commands in a multicommand rule?
When you mentioned a lack of timestamps in Snoopy I decided to try adding "timer" in front of each "rm -f *.o" in my little test makefile and got really strange results. In both cases I was booting with an SFS2 partition. Without timer in front of each command the total execution time was 8 seconds. With timer running each command the total time was 0.2 seconds. It doesn't make sense that adding a command to launch and time each individual command would be faster than just running the commands.
AmigaOne X1000 with 2GB memory - OS4.1 FE
Post Reply