NGFilesystem file corruption

AmigaOne X5000 platform specific issues.

NGFilesystem file corruption

Postby xenic » Wed Aug 16, 2017 12:22 am

While fixing some problems like USB Wilreless mouse failure and sound card issues, I've had to reboot a lot. Whenever I had to reboot with Odyssey still open, my Odyssey cookies got corrupted and I had to sign in again at all the sites where I have elected "remember me" (i.e. stay logged in). I've never had a problem like this on my X1000. I finally moved Odyssey to an SFS2 partition and the cookie file corruption when rebooting stopped. Presumably, Odyessy is keeping the cookies file open due to the constant updating of the file. I'm guessing that NGFilesystem reacts differently to a file not being closed before a reboot. Rebooting with Odyssey open never created problems for me with FastFilesystem or SFS1/2 on my X1000. Moving Odyssey to an SFS2 partition seems to have fixed the problem for me.

To reproduce:
Have Odyssey installed on an NGFilesystem.
Make sure the Odyssey prefs are set to save cookies.
Start Odyssey and open the Cookies window.
Unless your cookies file is already corrupt you should see all your cookies in the window.
Check your Conf/cookies file and note the size and date.
Visit some WEB sites and check the cookies file to be sure cookies have been written.
Wait until you're sure nothing is being written to a disk and perform a soft reboot.
After the system reboots, start Odyssey again and open the cookies window.
If the cookies file has been corrupted the window will be empty. No cookies shown.
If you check the cookies file, the date and filesize may be different that before you rebooted.

Can ayone else reproduce this problem??
AmigaOne X1000 with 2GB memory - OS4.1 FE
xenic
 
Posts: 1054
Joined: Sun Jun 19, 2011 1:06 am

Re: NGFilesystem file corruption

Postby tonyw » Wed Aug 16, 2017 12:38 am

Thanks for that report. I'll look into it (with the current version of NGFS).

As you can imagine, several advances have been made since that version was released (bug fixes, improvements).
cheers
tony
User avatar
tonyw
AmigaOS Core Developer
AmigaOS Core Developer
 
Posts: 1260
Joined: Wed Mar 09, 2011 2:36 pm
Location: Sydney, Australia

Re: NGFilesystem file corruption

Postby Spectre660 » Wed Aug 16, 2017 12:40 am

Spectre660
 
Posts: 1385
Joined: Sat Jun 18, 2011 3:16 pm
Location: Montserrat

Re: NGFilesystem file corruption

Postby xenic » Wed Aug 16, 2017 2:24 am

Spectre660 wrote:http://forum.hyperion-entertainment.biz/viewtopic.php?f=57&t=3586&start=10#p40884

That topic seems to confirm the problem. I investigated it for a day and at one point the cookies file showed up as one size with the list command but when I copied it to ram: it was actually a smaller size. AmigaDOS 'copy' command wouldn't copy it at all but returned an error about an invalid lock.
AmigaOne X1000 with 2GB memory - OS4.1 FE
xenic
 
Posts: 1054
Joined: Sun Jun 19, 2011 1:06 am

Re: NGFilesystem file corruption

Postby daveyw » Wed Aug 16, 2017 7:14 am

I haven't had many reboots, but I have noticed that Odyessy isn't always caching my logins. Couldn't remember if it ever did.... 8-)
User avatar
daveyw
 
Posts: 144
Joined: Mon Sep 12, 2011 8:44 pm

Re: NGFilesystem file corruption

Postby xenic » Wed Aug 16, 2017 7:01 pm

daveyw wrote:I haven't had many reboots, but I have noticed that Odyessy isn't always caching my logins. Couldn't remember if it ever did.... 8-)

That's how I discovered this problem. I was needing to sign in to this site and others every day which never happened when using an X1000 or SAM440.

Your logins are remembered (cached) with cookies If you select 'remember me' or 'stay logged in' or similar, The WEB site will set a cookie in your browser which the browser will save in it's cookie database. You need to have accept and save cookies set in the Odyssey Settings/Privacy window. Sites can't remember you if the cookies aren't saved.

It's easy to tell if your cookies have been corrupted. If you open the Odyssey Cookies window and it's empty after visiting a number of WEB sites, then the cookies file is probably corrupt. The only way to fix it is to delete the Odyssey/Conf/cookies.db file.
AmigaOne X1000 with 2GB memory - OS4.1 FE
xenic
 
Posts: 1054
Joined: Sun Jun 19, 2011 1:06 am

Re: NGFilesystem file corruption

Postby xenic » Wed Aug 16, 2017 9:05 pm

tonyw wrote:Thanks for that report. I'll look into it (with the current version of NGFS).

If you read the 'NGFilesystem' topic in this forum you will see that this problem might go beyond Odyssey and affect other programs that use SQL files.
AmigaOne X1000 with 2GB memory - OS4.1 FE
xenic
 
Posts: 1054
Joined: Sun Jun 19, 2011 1:06 am

Re: NGFilesystem file corruption

Postby tonyw » Thu Aug 17, 2017 2:05 am

Thanks to xenic's help, I can now confirm the behaviour. It is caused by the fact that NGFS "writes behind" its meta data. File data is written immediately, but meta data (eg the file header, containing the file size) is not updated until the next write or the file is closed. So, if the system is rebooted before closing the file, some data will be lost.

I am still not sure whether to treat this as a bug or not.
cheers
tony
User avatar
tonyw
AmigaOS Core Developer
AmigaOS Core Developer
 
Posts: 1260
Joined: Wed Mar 09, 2011 2:36 pm
Location: Sydney, Australia

Re: NGFilesystem file corruption

Postby tonyw » Thu Aug 17, 2017 7:29 am

This is the sort of behaviour that you would expect - reboot the system with files open for Write and you are likely to lose data. The journalling feature ensures that the disk contents remain consistent.

Since it isn't explicitly defined by the DOS API, it is not surprising that one file system behaves differently from another, especially when one of them is a journalling FS and the others are not.

If I were to suggest any changes here, they would go along these lines:
1) Operator should not reboot system and expect data in open files to be preserved.
2) Odyssey could flush data to its cookie file after making changes, in case of crashes or reboots, but it should not be blamed for (1).

I shall not be "fixing" this.
cheers
tony
User avatar
tonyw
AmigaOS Core Developer
AmigaOS Core Developer
 
Posts: 1260
Joined: Wed Mar 09, 2011 2:36 pm
Location: Sydney, Australia

Re: NGFilesystem file corruption

Postby xenic » Thu Aug 17, 2017 3:52 pm

tonyw wrote:This is the sort of behaviour that you would expect - reboot the system with files open for Write and you are likely to lose data. The journalling feature ensures that the disk contents remain consistent.

Since it isn't explicitly defined by the DOS API, it is not surprising that one file system behaves differently from another, especially when one of them is a journalling FS and the others are not.

If I were to suggest any changes here, they would go along these lines:
1) Operator should not reboot system and expect data in open files to be preserved.
2) Odyssey could flush data to its cookie file after making changes, in case of crashes or reboots, but it should not be blamed for (1).

I shall not be "fixing" this.

In that case I shall be using SFS2 instead :-)
I already expressed my opinion in a private email.

EDIT: With reference to item 2 in your post: I used Snoopy to check what IO functions Odyssey is using. It's using unbuffered AmigaDOS 'Read' and 'Write' commands for cookies so I don't think there's any flushing to be done.
AmigaOne X1000 with 2GB memory - OS4.1 FE
xenic
 
Posts: 1054
Joined: Sun Jun 19, 2011 1:06 am

Next

Return to Platform: AmigaOne X5000

Who is online

Users browsing this forum: No registered users and 1 guest