Jump to content
DerelictStudios Forums
jonwil

My Work On C&c3 So Far

Recommended Posts

In terms of the overall structure of the *.big file, it is the same as BFME2, Generals etc. The difference is that files inside it can be compressed. So far I found one algorithm, a custom EA compression (also used for medal of honor somewhere) called refpack. There is also mention of the use of zlib compression (the same compression used on a *.zip file and in many other places) but as of yet I haven't seen it being used.

 

Attached to this post is an executable file and source code for a command line program to decompress a refpack compressed file (extracted with a program such as XCC mixer), just run the program to figure out how its used. Hopefully it will be possible to add this decompression to XCC mixer and other such programs directly.

 

The source code in the zip file is licensed under the GNU General Public License. If you use it in an existing open source program, thats fine (although I would like credit for it and also details of any changes to the code). If you wish to use it in a program that is not open source, please let me know and I will consider letting you use it.

 

Now onwards to individual files inside the big file. Any files with a TGA extension are standard truevision Targa files (familiar to anyone modding Generals, BFME etc)

any file with an ini extension is a normal ini file

 

the c&c3.str file contains all the strings used in the game. It appears to just be a text file. I think str files were used/supported in previous games (e.g. Generals) but I am not sure.

 

WND files are used for some purpose in the GUI I think and are text files.

 

VP6 files are movie files (IIRC BFME uses the same movie format)

 

XML files are, well, XML files.

 

MAP files are most likely the maps. They appear to be compressed with some form of compression. What they decompress to is unclear at this point.

 

Related to the scripts are the following files:

scriptevents.xml (XML data for script events)

scriptevents.xsd (XML schema file for scriptevents.xml)

scriptevents.xsx (some file left over from the XML schema designer program they used)

scripts.ini (some kind of ini file for the scripts)

scripts.lua (from what I can tell, C&C3 is using the LUA scripting language for scripts and this file defines some kind of base functions for it)

 

There are 2 other kinds of files I will talk about in a future post once I understand more about them:

cdata files

and the filesets which have a .bin file, a .imp file, a .manifest file and a .relo file all with the same name

decompress.zip

Share this post


Link to post
Share on other sites

Hello jonwil, phoib!

 

jonwil, the program couldn't start so i fired up MSVC++6 and tried recompiling.

The compiler returned 5 errors - redifinition of "i". I removed the "unsigned int" part and defined "unsigned int i=0;" right after "#include <stdio.h>".

 

Here is the modified package. Source included.

 

The program works, however i still can't find the unit definitions their models or textures. Could it be that they've merged the model with the corresponding unit's ini file and texture? They did say the new model format is text-editable...but i didn't think they'd go as far as smacking the texture in the file as well <_<

 

I wish i could do more, but my C++ "skills" are very limited. Perhaps someone could modify this to scan the directory that it's located in and make it convert all files in the directory? That would be a relief, since converting file-by-file is a pain in the ass...

decompress2.zip

Edited by Mad Ivan

Share this post


Link to post
Share on other sites

I send Olaf regular updates of this topic, no worries :)

 

I should get a hold of Matze...

 

And, all data are now compressed, so even with texteditable models, it means that they are apparently exported to W3x, the xml format, and those are compressed to the cdata. I guess they applied the same algorythm to the DDS, really...

Share this post


Link to post
Share on other sites

I'm interested in the music and sound files.

So far, those haven't showed up in the .big files but I would guess that they might be hiding in one of those .bin files.

 

Here's to hope-ing there will be a way to have a look inside the bin files..

Share this post


Link to post
Share on other sites

I'm betting the music's in that "static_demo_common.bin" file somewhere. I've tried a handful of extractors on these things and they won't pop open, so they're not a standard .bin format as far as I can tell.

 

EDIT: Nor are they simply renamed rar/zip files - didn't expect them to be, but sometimes it's worth a shot. It has worked on several occasions.

 

~~Put on a quick text output from UCC that may shed some light on the situation.

 

The decoding programs didn't accept the .bin files, so they may be using a different compression, or may not be compressed.

 

EDIT3: Sharper viewers may notice I again tried to change the extension in the screenshot - to no avail, unfortunately. :rolleyes:

post-152-1172717561.jpg

Edited by Arcalane

Share this post


Link to post
Share on other sites

Full VS 2005 project, some code annotation, some changes, x86 & amd64 binaries...

 

Untested atm, though... my changes my have broken it for all I know.

 

I'll see if I can document the decompression function some more...

 

Edit: Quick change to get linker version setting working...

 

Edit: removed, updated below...

Edited by Chao7

Share this post


Link to post
Share on other sites
I'm betting the music's in that "static_demo_common.bin" file somewhere. I've tried a handful of extractors on these things and they won't pop open, so they're not a standard .bin format as far as I can tell.

 

EDIT: Nor are they simply renamed rar/zip files - didn't expect them to be, but sometimes it's worth a shot. It has worked on several occasions.

 

~~Put on a quick text output from UCC that may shed some light on the situation.

 

The decoding programs didn't accept the .bin files, so they may be using a different compression, or may not be compressed.

 

EDIT3: Sharper viewers may notice I again tried to change the extension in the screenshot - to no avail, unfortunately. :rolleyes:

The .bin files are compressed with refpack (the header matches). If you run it through a decompression program it might help.

Share this post


Link to post
Share on other sites
Full VS 2005 project, some code annotation, some changes, x86 & amd64 binaries...

 

Untested atm, though... my changes my have broken it for all I know.

 

I'll see if I can document the decompression function some more...

 

Edit: Quick change to get linker version setting working...

Found a small bug.

int __fastcall GetUncompressedSize( const unsigned char *buffer )
{
    unsigned short headerflags = InvertOrderOfBytes( buffer, 2 );

    if ((headerflags & 0x3EFF) == 0x10FB)
    {
 if (headerflags & 0x8000)
 {
     return InvertOrderOfBytes( buffer + 2, 4);
 }
 return InvertOrderOfBytes( buffer + 2, 3);
    }
    return 0;
}

The initial GetUncompressedSize was missing the returns in the inner if statements, so it would always assume it was uncompressed. I changed those, and it works fine on the .bin files. The EXE file also didn't work for me at all. I tweaked the project settings to run the proper command line, and I can run from inside VS.

Edited by mastermind

Share this post


Link to post
Share on other sites

Yeah, copy and paste getting out of hand, fixed...

 

Error messages changed, some inserted...

 

The command line you changed you're referring to is for debugging/execution right?

 

Yeah, you have to change that, sorry.

 

Edit: Namespace added & factored, packing problem fixed, somewhat tested...

 

Edit2: Removed, see next attachment...

Edited by Chao7

Share this post


Link to post
Share on other sites

I haven't found anything interesting yet. The .manifest files look somewhat interesting (I found another decompression program that works on the manifests, but not that .bin file), but I can't figure out any sort of format to it. The .cdata files also look like some sort of new binary format, but so far I can't figure anything out beyond the apparent header, which is AC B9 12 93 in hex. I would expect to find other bits that would suggest a header, but I can't seem to find them. There are a few things that seem close to the file size, but none are really that close.

Share this post


Link to post
Share on other sites

Well, I've sucessfully extracted DDS files from some of the .cdata files.

 

The DDS files aren't actually compressed, but they have a bunch of undecipherable data preceding the DDS header. Removing the data at the beginning of the file will allow the DDS file to be opened in photoshop (or whatever).

 

I wrote a small program which automatically scans and decodes all .cdata files in the folder. If no DDS header is found, the file is skipped (usually).

 

EDIT: It appears that the static_demo_common folder in the .big is comprised exclusively of DDS files.

CnC3_Decompressor.zip

Edited by Sc4Freak

Share this post


Link to post
Share on other sites

Could you post a copy of the decompressed manifest file?

 

For some reason, memcpy throws a rung where joe-for-loop doing the same works fine...

 

Well, it finishes anyway, don't know if it's "fine"... need to see compare the contents. Something else I changed may have borked it.

Edited by Chao7

Share this post


Link to post
Share on other sites
Well, I've sucessfully extracted DDS files from some of the .cdata files.

 

The DDS files aren't actually compressed, but they have a bunch of undecipherable data preceding the DDS header. Removing the data at the beginning of the file will allow the DDS file to be opened in photoshop (or whatever).

 

I wrote a small program which automatically scans and decodes all .cdata files in the folder. If no DDS header is found, the file is skipped (usually).

 

EDIT: It appears that the static_demo_common folder in the .big is comprised exclusively of DDS files.

The data preceeding it is a header. It's 40 bytes, with a magic number of ACB91293. The next 4 bytes are different for each file, but have an unknown purpose. Next is another common 4 byte sequence with values DC5EEE9D. There another 4 unknown bytes, and then the file size of the DDS file (it is 40 bytes smaller than the full file size for the header). The remaining header bytes appear to be unused.

Share this post


Link to post
Share on other sites
Could you post a copy of the decompressed manifest file?

 

For some reason, memcpy throws a rung where joe-for-loop doing the same works fine...

 

Well, it finishes anyway, don't know if it's "fine"... need to see compare the contents. Something else I changed may have borked it.

Attached if IPB likes it.

static_demo_common.manifest.out

Share this post


Link to post
Share on other sites

Hmm, that does appear the same. That's what I get for looking at 3 files from the same alphabetic section. The location of the file size is still right though, or at least I think it is.

:P

 

Well, now just to figure out what the .bin, .manifest, and .relo files mean...

Edited by mastermind

Share this post


Link to post
Share on other sites

Yeah that's what I get... can only replace some loops with memcpy, then.

 

Maybe the compiler has some kind of funny pointer logic problem...

 

So this decompressor works now... yay.

 

The manifest describes those files...

 

Textures, W3DCollisionBoxes, W3DMeshes, to AIPersonalityDefinition, to 'DATA' XML... they're all hash mapped or something.

CNC3Decompress_03_01_2007e.7z

Edited by Chao7

Share this post


Link to post
Share on other sites
Hmm, that does appear the same. That's what I get for looking at 3 files from the same alphabetic section. The location of the file size is still right though, or at least I think it is.

:P

 

Well, now just to figure out what the .bin, .manifest, and .relo files mean...

What about the rest of the cdata files in the global_demo_common folder? All we know is that the cdata files in the static_demo_common folder are DDS textures.

Share this post


Link to post
Share on other sites
The manifest describes those files...

 

Textures, W3DCollisionBoxes, W3DMeshes, to AIPersonalityDefinition, to 'DATA' XML... they're all hash mapped or something.

 

The files named after the folder act as an index for those hexadecimal named cdata files... the manifest contains their filenames, and description (?)...

Edited by Chao7

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×