jonwil 0 Report post Posted March 1, 2007 The main things we still need right now to proceed with hacking C&C3: 1.Some way to unpack the .cdata, .manifest, .bin, .relo and .imp files back into raw readable data such as textures, audio files, 3d models and XML game configuration files (that or the source files from EA for them) 2.A way to create these files again (assuming the game requires them to function or requires them to function at an acceptable speed :) I think those 2 things are the main things that we require at this point On top of that we need tools and examples and stuff for manipulating the new w3x 3d model files :) Share this post Link to post Share on other sites
Count von Phoib 0 Report post Posted March 1, 2007 Ok... Trying to list what we have right now, edit at will. Data\aptui\/ Has all the UI files apparently. All files appear in 4 forms, bin, imp, manifest, and relo. Manifest seems to be a linker file of sorts? Bin seems to be the main data, with relo consisting almost exclusively of FF FF FF FF etc. Data\global_demo_common\cdata\ Froniki believes these are all audio files... but that sure is a huge audio list! Anyone knowing more? Data\static_demo_common\cdata DDS files. This one is pretty clear. Data\maps\official\ Maps probably. All maps have 6 files: bin, imp, manifest, relo, who all have the map_demo name, and the tga + map file, who have the directory name (the tga file has the _art extension though). Data\ Has some files which function is unclear to me. Ok... Lets fill in a bit more blanks, shall we? :) Share this post Link to post Share on other sites
Arcalane 0 Report post Posted March 2, 2007 The .bin files probably aren't your standard .bin cabinet file, that's all I know. :P Although the imp, manifest and relo files may well be the key to cracking these things open. Maybe. Share this post Link to post Share on other sites
jonwil 0 Report post Posted March 2, 2007 basically a .bin, .manifest, .relo, .imp set goes together. It looks like they together make some kind of container which holds various "chunks" of data (a chunk might be a dds file, it might be something else) Share this post Link to post Share on other sites
Arcalane 0 Report post Posted March 2, 2007 basically a .bin, .manifest, .relo, .imp set goes together. It looks like they together make some kind of container which holds various "chunks" of data (a chunk might be a dds file, it might be something else) That's what I was figuring. I'm also figuring that the .bin files are semi-useless without the other manifest/relo/imp files - at least in terms of maps - but there aren't any files of those types for the static/global common data, so meh. :blink: Share this post Link to post Share on other sites
Froniki 0 Report post Posted March 2, 2007 (edited) about Data\static_demo_common\cdata found this in static_demo_common.manifest ACB9129387414E6EDC5EEE9D8608383A -> OnDemandTexture:IndividualImages_001 -> 9312B9AC.9DEE5EDC.6E4E4187.3A380886.cdata ACB9129339FE8D18DC5EEE9D8A25160B -> OnDemandTexture:IndividualImages_002 -> 9312B9AC.9DEE5EDC.188DFE39.0B16258A.cdata ACB91293762E5E92DC5EEE9DFBECA2D5 -> OnDemandTexture:IndividualImages_003 -> 9312B9AC.9DEE5EDC.925E2E76.D5A2ECFB.cdata ACB912932F3DBC69DC5EEE9D7329A7E5 -> OnDemandTexture:IndividualImages_004 -> 9312B9AC.9DEE5EDC.69BC3D2F.E5A72973.cdata ACB91293EE540865DC5EEE9D06AD0B6B -> OnDemandTexture:IndividualImages_005 -> 9312B9AC.9DEE5EDC.650854EE.6B0BAD06.cdata ACB91293F4137D42DC5EEE9D20CECB26 -> OnDemandTexture:IndividualImages_006 -> 9312B9AC.9DEE5EDC.427D13F4.26CBCE20.cdata ACB91293ABC24841DC5EEE9D2776D546 -> OnDemandTexture:IndividualImages_007 -> 9312B9AC.9DEE5EDC.4148C2AB.46D57627.cdata ACB91293FE3C1CBADC5EEE9D4861BF63 -> OnDemandTexture:IndividualImages_008 -> 9312B9AC.9DEE5EDC.BA1C3CFE.63BF6148.cdata ACB91293EC2691AFDC5EEE9D75A49B2F -> OnDemandTexture:IndividualImages_009 -> 9312B9AC.9DEE5EDC.AF9126EC.2F9BA475.cdata ACB9129324475D89DC5EEE9D9132EC0A -> OnDemandTexture:IndividualImages_010 -> 9312B9AC.9DEE5EDC.895D4724.0AEC3291.cdata ACB912939AC3862DDC5EEE9D23603175 -> OnDemandTexture:IndividualImages_011 -> 9312B9AC.9DEE5EDC.2D86C39A.75316023.cdata ACB91293CD299934DC5EEE9DC62B7F1D -> OnDemandTexture:IndividualImages_012 -> 9312B9AC.9DEE5EDC.349929CD.1D7F2BC6.cdata ACB91293A30A9F98DC5EEE9D06F7DC72 -> OnDemandTexture:IndividualImages_013 -> 9312B9AC.9DEE5EDC.989F0AA3.72DCF706.cdata ACB91293EC5F0A5CDC5EEE9D7F67B07E -> OnDemandTexture:IndividualImages_014 -> 9312B9AC.9DEE5EDC.5C0A5FEC.7EB0677F.cdata ACB91293E2735379DC5EEE9DD7ACAA44 -> OnDemandTexture:IndividualImages_015 -> 9312B9AC.9DEE5EDC.795373E2.44AAACD7.cdata ACB912931F039B14DC5EEE9DF52B22EA -> OnDemandTexture:IndividualImages_016 -> 9312B9AC.9DEE5EDC.149B031F.EA222BF5.cdata ACB91293CE39CCA4DC5EEE9D16C4D30F -> OnDemandTexture:IndividualImages_017 -> 9312B9AC.9DEE5EDC.A4CC39CE.0FD3C416.cdata ACB912937DD297BDDC5EEE9DE03F75BA -> OnDemandTexture:IndividualImages_018 -> 9312B9AC.9DEE5EDC.BD97D27D.BA753FE0.cdata ACB912939CB9EFA2DC5EEE9D5C0C7823 -> OnDemandTexture:IndividualImages_019 -> 9312B9AC.9DEE5EDC.A2EFB99C.23780C5C.cdata ACB912937F83C46BDC5EEE9DBB9DDACB -> OnDemandTexture:IndividualImages_020 -> 9312B9AC.9DEE5EDC.6BC4837F.CBDA9DBB.cdata ACB912934BB14E61DC5EEE9D2FC65642 -> OnDemandTexture:IndividualImages_021 -> 9312B9AC.9DEE5EDC.614EB14B.4256C62F.cdata ACB912938CD06513DC5EEE9DBEC1129B -> OnDemandTexture:IndividualImages_022 -> 9312B9AC.9DEE5EDC.1365D08C.9B12C1BE.cdata ACB91293125BCB91DC5EEE9D378A002C -> OnDemandTexture:IndividualImages_023 -> 9312B9AC.9DEE5EDC.91CB5B12.2C008A37.cdata ACB912931730055EDC5EEE9DED4D704F -> OnDemandTexture:IndividualImages_024 -> 9312B9AC.9DEE5EDC.5E053017.4F704DED.cdata ACB9129331E32272DC5EEE9D4994D768 -> OnDemandTexture:IndividualImages_025 -> 9312B9AC.9DEE5EDC.7222E331.68D79449.cdata ACB91293CCD5E056DC5EEE9DB9B5414E -> OnDemandTexture:IndividualImages_026 -> 9312B9AC.9DEE5EDC.56E0D5CC.4E41B5B9.cdata ACB91293CD2E9F58DC5EEE9DB2406EB5 -> OnDemandTexture:IndividualImages_027 -> 9312B9AC.9DEE5EDC.589F2ECD.B56E40B2.cdata ACB912930A92F6FFDC5EEE9D5FCE5EB3 -> OnDemandTexture:IndividualImages_028 -> 9312B9AC.9DEE5EDC.FFF6920A.B35ECE5F.cdata ACB912931C97DF8BDC5EEE9D793B592E -> OnDemandTexture:IndividualImages_029 -> 9312B9AC.9DEE5EDC.8BDF971C.2E593B79.cdata ACB912939386167DDC5EEE9D386A3B36 -> OnDemandTexture:IndividualImages_030 -> 9312B9AC.9DEE5EDC.7D168693.363B6A38.cdata ACB912935D89C058DC5EEE9D41842A20 -> OnDemandTexture:IndividualImages_031 -> 9312B9AC.9DEE5EDC.58C0895D.202A8441.cdata ACB9129366E40F75DC5EEE9D2D080738 -> OnDemandTexture:IndividualImages_032 -> 9312B9AC.9DEE5EDC.750FE466.3807082D.cdata ACB91293987EF1A1DC5EEE9D2482DC3E -> OnDemandTexture:IndividualImages_033 -> 9312B9AC.9DEE5EDC.A1F17E98.3EDC8224.cdata ACB9129362BE6D5BDC5EEE9DD519318C -> OnDemandTexture:IndividualImages_034 -> 9312B9AC.9DEE5EDC.5B6DBE62.8C3119D5.cdata ACB912938FE6D471DC5EEE9D13C7E0A9 -> OnDemandTexture:IndividualImages_035 -> 9312B9AC.9DEE5EDC.71D4E68F.A9E0C713.cdata ACB91293AD52EB4EDC5EEE9D778BE9B5 -> OnDemandTexture:IndividualImages_036 -> 9312B9AC.9DEE5EDC.4EEB52AD.B5E98B77.cdata ACB912931388E86BDC5EEE9D8B39B7B0 -> OnDemandTexture:IndividualImages_037 -> 9312B9AC.9DEE5EDC.6BE88813.B0B7398B.cdata Edited March 2, 2007 by Froniki Share this post Link to post Share on other sites
jonwil 0 Report post Posted March 2, 2007 Froniki, how did you find out that mapping between the name and the .cdata files? Is it possible to do the same for the global_demo_common files? Share this post Link to post Share on other sites
Froniki 0 Report post Posted March 2, 2007 global_demo_common.manifest 4D086B164BBBDB98770F4146741EAFAD -> AudioFile:ABLight_weapFirea -> 166b084d.46410f77.98dbbb4b.adaf1e74.cdata 4D086B16 -> 166b084d 4BBBDB98 -> 98dbbb4b 770F4146 -> 46410f77 741EAFAD -> adaf1e74 all files 166b084d.46410f77.xxxxxxxx.xxxxxxxx.cdata in manifest looks like this 4D086B16xxxxxxxx770F4146xxxxxxxx -> AudioFile: xxxxxxxxxxxxx Share this post Link to post Share on other sites
kmx 0 Report post Posted March 2, 2007 (edited) Hey, .manifest files: 00010500 [TAG#1] C3CC2037 [TIMESTAMP] CFBC4AF5 [TAG#2] 6C260000 [VIRTUAL FILE COUNT] 641E1000 [bIN SIZE] 6C0D0000 [RELO SIZE] E0000000 unk 78060000 unk B03D0100 [sTRING INDEX SIZE] 00000000 unk 02710400 [VIRTUAL FILENAMES LIST SIZE (strs like: AudioEvent:GDI_IonCannon_Laser1) 72010000 [REAL FILENAMES LIST SIZE (stra like: DATA:sounds/ambientstream.xml...etc)] ...filerecord SIZE = FILE_COUNT * 0x2C 93964A56 155B622F E3CB9F99 2FD82410 [FILE HASH, also part of the [CONTAINER_NAME]\cdata\XXXXXXXX.XXXXXXXX.XXXXXXXX.XXXXXXXX.cdata 00000000 00000000 00000000 00000000 28000000 10000000 00000000 ...STRING INDEX 2 dwords per item ...VIRTUAL FILENAMES LIST null terminated strings ...REAL FILENAMES LIST null terminated strings Edited March 2, 2007 by kmx Share this post Link to post Share on other sites
jonwil 0 Report post Posted March 2, 2007 So assuming all this is correct and the files under global_demo_common really are audio files, we need to find out their format. They could be compressed (with MP3?) or they could be raw sound data. Playing around with IDA hasn't shown me where the MP3 decompression routine is, nor what the game does with those files that are supposedly audio files (I cant even find where the code "opens" the file and reads its contents) Share this post Link to post Share on other sites
Mad Ivan 0 Report post Posted March 2, 2007 So assuming all this is correct and the files under global_demo_common really are audio files, we need to find out their format. They could be compressed (with MP3?) or they could be raw sound data. Playing around with IDA hasn't shown me where the MP3 decompression routine is, nor what the game does with those files that are supposedly audio files (I cant even find where the code "opens" the file and reads its contents) If this will help - RA2 used IMA ADPCM Wav files...maybe it's the same here? Share this post Link to post Share on other sites
kmx 0 Report post Posted March 2, 2007 (edited) Some of \data\global_demo_common\cdata\*.cdata files are RAW-audio with 8 byte header. 04 TAG 00 unk (0=1 channel, 1=2 channels?) BB80(48khz) 0000AD00 (time) 000066C0 (file size - 8) 0000AD00 (time again) Edited March 2, 2007 by kmx Share this post Link to post Share on other sites
jonwil 0 Report post Posted March 2, 2007 (edited) ok, so, can someone write a program to convert that raw audio into something one can listen to? And/or can someone provide instructions as to how to read it in with the "read raw data" option in a program like audacity? (I cant seem to get it to read in that program regardless of the options I try to pass it). Which option do I pass for the data type? 16 bit PCM? 8 bit PCM? 32 bit PCM? What about endianness? And start offset? 8? Also, what about other files that aren't these audio files? Someone should make a list of those and try and match them to entries in the manifest to see if that sheds any light on what they could be... Based on what I can see, I suspect that there is no such thing as an "decompiler" or "unpacker" for these stream files. Most likely what happens is that the people doing C&C3 work make changes and then run some kind of "build program" which builds a new set of files with all the work being done on the source files. So what we will need is some kind of source files (either 100% of the source files or enough of the source files to mod the game by using the other existing files as they are) and the tools to rebuild this data. I have a strong feeling that these files are intended as an optomization to load the game faster (rather than have the game parse the ini or xml file into an internal structure every time its loaded, it can be pre-parsed once with the build tools or whatever and then loaded much faster. Also, note the string "Contact CnC3TechnicalArchitecture@ea.com" that has been left in the demogame.dat file :) Edited March 2, 2007 by jonwil Share this post Link to post Share on other sites
Count von Phoib 0 Report post Posted March 2, 2007 Yeah, they made everything into XML and then let them be pre-parsed for faster loading. It means that I don't have much faith that we can make a decompiler anytime soon, that decompiles all the files. I mean... has anybody even seen something that resembles XML files? Share this post Link to post Share on other sites
kmx 0 Report post Posted March 2, 2007 there are 2 types of sound files there: 8byte header+data like: AudioFile:WIRadio_clicksA23 or just data, like: AmbientStream:AmbStream_YellowZone01_Temperate_5point1 They are all 16bit PCM... but they are encrypted or maybe some codec was used after all... Share this post Link to post Share on other sites
jonwil 0 Report post Posted March 2, 2007 Could the audio in those audio files be MP3 audio? (in some or all files?) Share this post Link to post Share on other sites
jonwil 0 Report post Posted March 3, 2007 It is my opinion that it will not be possible to modify C&C3 without the source XML files (which EA will need to provide). Share this post Link to post Share on other sites
Arcalane 0 Report post Posted March 3, 2007 Could the audio in those audio files be MP3 audio? (in some or all files?) Well, they don't appear to be compressed as there was no filesize change (although the decompressors didn't spit them out saying they didn't recognise the compression...) but they're not mp3's. They may be some format of wav, because they were causing WinAMP to lock up. Maybe they're compressed multiple times. Share this post Link to post Share on other sites
jonwil 0 Report post Posted March 3, 2007 They are not WAV files, they do not contain any of the things a WAV file normally has. (i.e. headers, WAV signature etc). No signatures of any other known audio file types are visible either. Most likely these audio are one or more of the following: (different cdata files could be different formats) 1.Raw audio data compressed with MP3 compression algorithm (we do know C&C3 is using MP3 somewhere) 2.Raw audio data compressed in some other way that we haven't found yet or 3.Raw audio data in an uncompressed format What we need to do is to investigate all this and see if we can figure out a way to convert these files into something playable in an audio player like WinAmp. Share this post Link to post Share on other sites
Arcalane 0 Report post Posted March 3, 2007 Well, there's this in the playertemplate.ini (once decoded) that may point towards a solution, or at least give some ideas; LoadScreenMusic = TEMP_RAM_Music360_LoadScreen ; If you change this, remember it must be a RAM-based (no-stream) piece of music Share this post Link to post Share on other sites
Froniki 0 Report post Posted March 3, 2007 (edited) based on kmx info about manifest .manifest files: 00010500 [TAG#1] C3CC2037 [TIMESTAMP] CFBC4AF5 [TAG#2] 6C260000 [VIRTUAL FILE COUNT] 641E1000 [BIN SIZE] 6C0D0000 [RELO SIZE] E0000000 unk 78060000 unk B03D0100 [STRING INDEX SIZE] 00000000 [INCLUDE MANIFESTS LIST SIZE (strs like: static.manifest)] 02710400 [VIRTUAL FILENAMES LIST SIZE (strs like: AudioEvent:GDI_IonCannon_Laser1)] 72010000 [REAL FILENAMES LIST SIZE (stra like: DATA:sounds/ambientstream.xml...etc)] ...filerecord SIZE = FILE_COUNT * 0x2C 93964A56 155B622F E3CB9F99 2FD82410 [FILE HASH, also part of the [CONTAINER_NAME]\cdata\XXXXXXXX.XXXXXXXX.XXXXXXXX.XXXXXXXX.cdata 00000000 00000000 00000000 [VIRTUAL FILE ID][OFFSET from begining of VIRTUAL FILENAMES LIST] 00000000 [REAL FILE ID][OFFSET from begining of REAL FILENAMES LIST] 28000000 [FILESIZE in BIN file] 10000000 00000000 ...STRING INDEX 2 dwords per item ...VIRTUAL FILENAMES LIST null terminated strings ...REAL FILENAMES LIST null terminated strings Edited March 3, 2007 by Froniki Share this post Link to post Share on other sites
kmx 0 Report post Posted March 4, 2007 They are not WAV files, they do not contain any of the things a WAV file normally has. (i.e. headers, WAV signature etc). No signatures of any other known audio file types are visible either. Most likely these audio are one or more of the following: (different cdata files could be different formats) 1.Raw audio data compressed with MP3 compression algorithm (we do know C&C3 is using MP3 somewhere) 2.Raw audio data compressed in some other way that we haven't found yet or 3.Raw audio data in an uncompressed format What we need to do is to investigate all this and see if we can figure out a way to convert these files into something playable in an audio player like WinAmp. If you load any of those sound files, either normal or streams into program such as Sony Sound Forge 8, the program will tell you the predicted file size. In the program I've selected 48khz raw audio, 16bit PCM, 1 channel and starting offset 8 (or 0 for the streams) and the calculated raw data size matches perfectly! This means there is no compression used, however - it is not playable, so I'm pretty sure they've used some encryption on it. Share this post Link to post Share on other sites
Arcalane 0 Report post Posted March 4, 2007 If you load any of those sound files, either normal or streams into program such as Sony Sound Forge 8, the program will tell you the predicted file size. In the program I've selected 48khz raw audio, 16bit PCM, 1 channel and starting offset 8 (or 0 for the streams) and the calculated raw data size matches perfectly! This means there is no compression used, however - it is not playable, so I'm pretty sure they've used some encryption on it. Jeez, this is gonna be like jumping through flaming hoops! And here I thought getting my hands on the DoW OST was difficult. :blink: Share this post Link to post Share on other sites
