[ Wiki | Release Lists | Patches | J-DB ]
PocketHeaven Sponsors:
[ Jandaman | DealExtreme ]



Pocket Heaven
Pocket Emulation Discussion Boards
 
    SearchSearch   MemberlistMemberlist  RSS FeedRSS Feed  RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Unofficial PocketNES 9.98-k

 
Post new topic   Reply to topic    Pocket Heaven Forum Index -> PocketNES
View previous topic :: View next topic  
Author Message
kuwanger
Member


Joined: 11 Mar 2004
Posts: 288

PostPosted: Sun Sep 11, 2005 6:40 am    Post subject: Unofficial PocketNES 9.98-k Reply with quote

Warning - Alpha Release

Yep, I've decided to do another PocketNES version with compressed file support. There are some rather major changes to the design, though. Here's a list:

1. Only supports up to 128KB ROMs (to avoid all sorts of nasty stuff)
2. Now works with GBA LZ77 compressed files too (use extension .l77)
3. Only works with pogoshell (this is unlikely to change given how much of a hassle it is to getting compilations to work well with compressed files)

Having stated all that, this is my first go at reproducing compression support with PocketNES, so I'd assume there's going to be at least a few bugs. You can use the old pocketnes-k's nez_cmp program to compress files to .nez and my own d or other programs like dcmp and GBACrusher for .l77. They're not included with the binary+source.

Note: This is primarily a test release. I've not done extensive testing to see if save states and the like are broken (though I doubt they are). Please reply with any positive/negative comments.

Binary and Source
_________________
Kuwanger Projects Like stag beetles in tubes
Back to top
View user's profile Send private message
tepples
Big Bird


Joined: 11 Jul 2004
Posts: 3015
Location: NE Indiana, USA

PostPosted: Sun Sep 11, 2005 6:49 am    Post subject: Re: Unofficial PocketNES 9.98-k Reply with quote

kuwanger wrote:
Yep, I've decided to do another PocketNES version with compressed file support. There are some rather major changes to the design, though. Here's a list:

[...]
2. Now works with GBA LZ77 compressed files too (use extension .l77)
[...]

You can use the old pocketnes-k's nez_cmp program to compress files to .nez and my own d or other programs like dcmp and GBACrusher for .l77.

Will any old GBA-LZ77 compression program work? Does it have to be VRAM-safe?
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger MSN Messenger
Dwedit
Administrator


Joined: 26 Feb 2004
Posts: 2249
Location: Chicago!

PostPosted: Sun Sep 11, 2005 6:53 am    Post subject: Reply with quote

Apack compresses a lot better, maybe you could merge in some parts of pocketnes_compy into this?
_________________
Here come the fortune cookies! Here come the fortune cookies!
They're wearing paper hats!
Back to top
View user's profile Send private message Visit poster's website
kuwanger
Member


Joined: 11 Mar 2004
Posts: 288

PostPosted: Sun Sep 11, 2005 7:45 am    Post subject: Reply with quote

@tepples: Yea, any gba lz77 compression program will work. I was just listing a few.

@dwedit: Is there an open apack compressor out there? I'd rather not include decompression support when I can't legally include the compressor (and hopefully source). I need to port the lzo compressor, anyways, to java anyways to avoid the whole cross-platform compiling thing; if I include apack, I'd want to do the same.
_________________
Kuwanger Projects Like stag beetles in tubes
Back to top
View user's profile Send private message
kuwanger
Member


Joined: 11 Mar 2004
Posts: 288

PostPosted: Mon Sep 12, 2005 3:32 am    Post subject: lzo1x_999 Reply with quote

I've done some more testing. Using lzo1x_1 (the standard/only compressor in minilzo), I get an average of ~68.5% the original size. Using lzo1x_999 (from lzo), I get an average of ~60.7% the original size. Just so you know, all the lzo1x compressors are compatible. So, it'd seem the best thing to do would be to distribute the lzo1x_999 compressor.
_________________
Kuwanger Projects Like stag beetles in tubes
Back to top
View user's profile Send private message
Dwedit
Administrator


Joined: 26 Feb 2004
Posts: 2249
Location: Chicago!

PostPosted: Mon Sep 12, 2005 4:00 am    Post subject: Reply with quote

What's the maximum lookback size on those LZ compressors?
_________________
Here come the fortune cookies! Here come the fortune cookies!
They're wearing paper hats!
Back to top
View user's profile Send private message Visit poster's website
tepples
Big Bird


Joined: 11 Jul 2004
Posts: 3015
Location: NE Indiana, USA

PostPosted: Mon Sep 12, 2005 4:54 am    Post subject: Reply with quote

I can't say about the others, but GBA LZ77 uses a window of about 4 KB and runlengths of 18 bytes.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger MSN Messenger
Dwedit
Administrator


Joined: 26 Feb 2004
Posts: 2249
Location: Chicago!

PostPosted: Tue Sep 13, 2005 6:38 pm    Post subject: Reply with quote

Is there anything stopping someone from taking an existing LZ compressor (which uses a large window size) and generating an aplib compatible output stream?
_________________
Here come the fortune cookies! Here come the fortune cookies!
They're wearing paper hats!
Back to top
View user's profile Send private message Visit poster's website
tepples
Big Bird


Joined: 11 Jul 2004
Posts: 3015
Location: NE Indiana, USA

PostPosted: Tue Sep 13, 2005 10:00 pm    Post subject: Reply with quote

Dwedit wrote:
Is there anything stopping someone from taking an existing LZ compressor (which uses a large window size) and generating an aplib compatible output stream?

The UnRAR source code license prohibits use of the source code to learn the RAR data format in order to write a program that generates a RAR compatible bitstream. The aPlib unpacking source code license, on the other hand, doesn't explicitly prohibit this, so I guess the only thing is lack of time and/or lack of a generic LZ compressor. After reading the source code I can't seem to tell what's the largest supported window size.
_________________
-- Where is he?
-- Who?
-- You know, the human.
-- I think he moved to Tilwick.
Back to top
View user's profile Send private message Visit poster's website AIM Address Yahoo Messenger MSN Messenger
Dwedit
Administrator


Joined: 26 Feb 2004
Posts: 2249
Location: Chicago!

PostPosted: Tue Sep 13, 2005 10:48 pm    Post subject: Reply with quote

The window size is infinate. The gamma code used allows it to keep shifting left and adding 0/1 until it hits a zero bit indicating not to continue.

Pretty much once it knows that it's reading a gamma code, it follows this simple algorithm:
start with 2
* read a bit
* add the bit
* read another bit
* if 1, multiply by 2 and repeat
* if 0, the number is done.
So big numbers are represented in 2(n-1) bits.
_________________
Here come the fortune cookies! Here come the fortune cookies!
They're wearing paper hats!
Back to top
View user's profile Send private message Visit poster's website
kuwanger
Member


Joined: 11 Mar 2004
Posts: 288

PostPosted: Mon Sep 19, 2005 8:04 pm    Post subject: Reply with quote

I've been working on creating an apack compressor, based on the apack.s decompressor included with pocketnes_compy. So far, I've written a C version of apack.s, an apack compressor that doesn't work (unapack segfaults on trying to decompress what it produces; also, the output file is not optimal), and a pair of compressors/decompressors that are modelled after apack but are simplified and I've verified that work. This last pair I call kpack/unkpack. They're more a stop-gap until a proper apack compressor is built, to better understand how apack works. So, I'd appreciate any help on getting my apack.c program to work well, let alone at all.

What I've got so far: Binary and Source

Hopefully we will get an apack compressor working soon, and then I can properly incorporate the compressor into my pocketnes-k package. This of course assumes I can get that working properly. Smile
_________________
Kuwanger Projects Like stag beetles in tubes
Back to top
View user's profile Send private message
Dwedit
Administrator


Joined: 26 Feb 2004
Posts: 2249
Location: Chicago!

PostPosted: Wed Sep 21, 2005 1:27 am    Post subject: Reply with quote

There's also the big guns: LZMA. It's what 7-Zip uses. Appearently, it only requires 16k/32k (getting contradictory numbers here) of additional memory to decompress a file.
I think an easy way to get 32k of memory would be to use GBA_VRAM, and blank the screen during decompression, then restore the original content afterwards.

It might be really slow though, haven't speed tested it yet.
Edit: The docs suggest it would take about 3 seconds to decompress 256k. Still haven't speed tested it yet.
_________________
Here come the fortune cookies! Here come the fortune cookies!
They're wearing paper hats!
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    Pocket Heaven Forum Index -> PocketNES All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2005 phpBB Group