Does anybody know how to add more music tracks and or remove some of the music tracks that are present in the game?

Not that I am aware of, which is a shame as I would love to put the music from Tropico 1 into Tropico 3.

Not that I know of either, but I wish! I have to say, the soundtrack is lacking. Most of what they have is great, but there isn't enough variety for a game like this. I've been turning off the music and playing my own during recent games.

I too, would love to have a little program that reads the music files out of my existing Tropico discs (or installed games) and merge them into Tropico 3. Variety is the spice of life.

still noone got a hold on "hpack"? its a tool that is believed to be able to pack hpk files. with that and the "hpk extractor", found on haemimont forums, you should be able to exchange the .ogg files, at least if you create soundfiles with the exact same sizes.

(27-11-2009 01:48 AM)tropicotim Wrote:  still noone got a hold on "hpack"? its a tool that is believed to be able to pack hpk files. with that and the "hpk extractor", found on haemimont forums, you should be able to exchange the .ogg files, at least if you create soundfiles with the exact same sizes.

Thank you. I like the music that came with Tropico3 - I was just hoping to add to the selection. What I want is a way for the current engine to see and play all the music files that have been shipped with the various versions of Tropico over the years.

I'd like to copy the .xdt files from my old versions, and add them to the new version. Apparently, the .xdt files are in a weird format.

From what you say, if I were to get, say Flamenco music off of Jamendo in .ogg format, with the hpack tools, I could replace the current music. Do we know if we can add new music and keep the current music?

Thanks!

(29-11-2009 06:51 PM)degrees57 Wrote:  ... I'd like to copy the .xdt files from my old versions, and add them to the new version. Apparently, the .xdt files are in a weird format. ...

That has been discussed at Cafe Tropico.

(29-11-2009 07:43 PM)CoconutKid Wrote:  

(29-11-2009 06:51 PM)degrees57 Wrote:  ... I'd like to copy the .xdt files from my old versions, and add them to the new version. Apparently, the .xdt files are in a weird format. ...

That has been discussed at Cafe Tropico.

Is it just me, or are the forums at Cafe Tropico (@strategyplanet) down?

'hpack' works? (where, I never found one)?......Cafe Tropico is down for the count.

Unpacking the files will not work unless you can re-pack them. exactly. ASFAIK.

(30-11-2009 03:21 AM)Pi-Xerxes Wrote:  ......Cafe Tropico is down for the count.

No it's not! It moved.

http://www.the-nextlevel.com/tropico/cafe/index.php

(29-11-2009 09:08 PM)degrees57 Wrote:  Is it just me, or are the forums at Cafe Tropico (@strategyplanet) down?

As Coffeebean says, the Cafe has moved. Strategy Planet dumped a number of forums they used to host.

http://www.the-nextlevel.com/tropico/cafe/index.php

The Next Level is the new host.

---

A snip from an old (about T1) email in my old (five years ago) file:

Quote:Yep yep, according to the discussions on the subject (as well as from Phil himself), the MAC version indeed used MP3 format music files vs those infernal XDT proprietary compressed files for the Windows version. Do a search through the old messages over at the Cafe for MP3, there was quite a bit of discussion there eons ago on the subject (always drove me crazy that the MAC version used MP3 files).

(21-11-2009 08:52 PM)topleya Wrote:  I would love to put the music from Tropico 1 into Tropico 3.

That was my first thought upon loading the game. I so miss T1s music.

so far, i've been able to successfully replace Tropico 3's music. i also know how to change the filenames of the music files, but that's not particularly helpful unless i can also figure out how to add new music files--that whole compiled LUA script and all. i'll keep working on it, though.

but to replace the music files, just put an OGG format audio file into "tropico 3\Music" and rename it to TrackN.ogg where N is a number 1 to 15. this will override the music files in the Music.hpk archive. Simple if you already have some OGGs. if you need to create OGGs from MP3s, try LAME plus LameDropXPd or Razorlame to get the MP3 to WAV, then OGGdropXPd to get the WAV to OGG. if you want to know what music files to replace, you'll have to give the extracted original files a listen. get one of the HPK extractors floating around and extract "tropico 3\Packs\boot\Music.hpk".

on adding new music, thus far i know that the "Data\Sounds\Playlists\Playlists.lua" file extracted from Data.hpk controls what files are played. the filenames are easy enough to edit: at the end of the file, they're in a string resource block where each string is formatted thus:

• byte 04
• integer of the size of the string (plus terminating null, little-endian)
• the string itself
• terminating null byte
  

in the beginning of the playlists file, i know there's a pattern that corresponds to the number of tracks (15) but i can't quite figure out the original code or what to modify. my intuition says it's probably a random number put into a switch-case statement, though.

here's what i've found so far in the code in case anyone better at asm wants to take a look:

I think what you picked up on there is a list of static data.


Just before that static list is a count of how many items exist in the list. In the case of Playlists.lua, it is 33.

There clearly is a structure before that as well. It is a good start, nevertheless.

Edit: The structure before it appears to be byte-instructions. It will take a lot of work to decompile it.

(06-02-2010 05:28 AM)FordGT90Concept Wrote:  It will take a lot of work to decompile it.

I did it. I decompiled 37 unique operations out of the 126 in total. My results from that follow. Since I had to do this manually (none of the decompilers work apparently), the format is a little peculiar. But it basically reads as follows: BinaryCode: (Operation A B C) Assembler; // Psuedocode

Here's what that would look like in psuedocode:

What you'll notice is that the global variable AllTracks holds all the track filenames. Also, the last two blocks of code aren't needed for the full version of the game and the randomization must be done in the calling function. So basically, as long as you create a global array called AllTracks with as many elements as tracks you want, the game will take your new track listing.

Each element in the AllTracks array needs to be an array with the following keys:

• path - the path to the track relative to the tropico 3 directory. may also take absolute paths?
• frequency - guessing this is a weight for how likely it is that the track is to be picked, but i haven't tested it
• SilenceDuration - length of time in milliseconds of silence after the track has played?
• SilenceChance - chance that there will be silence after the track is played?
  

You can use the following LUA script to get you started (it includes Tropico 1 filenames for if you converted those to OGG):

After you've edited the above script, adding or removing whatever tracks you want, you'll need to save and compile it so the game can read it. Unfortunately, the default LUA compiler won't work because it uses doubles for number data where Tropico 3 uses integers. So you'll have to download my special LUA compiler with that modification made. Extract the luac.exe file to wherever you saved your script to (desktop in the example below). Then to compile, run "cmd" and follow this procedure, making sure to substitute appropriate drive letters, directories, and filenames where underlined:

• D:\Documents and Settings\User>C: #change current drive
• C:\>cd Documents and Settings\User\Desktop #change current directory
• C:\Documents and Settings\User\Desktop>luac -o "C:\Program Files\Valve\Steam\SteamApps\common\tropico 3\Data\Sounds\Playlists\Playlists.lua" Playlists.lua #compile the script and put the output in the correct directory
  

Then you should be all set. If you get a compile error, go back and check your syntax in the script. If you compile okay but get no music when starting a game, double check your script (it may have compiled okay, but the game still had troubles reading the array it output) and make sure your music files exist in the right place.

Good luck and happy modding!

Um, you gonna make an app to compile/decompile LUA files?

Also, what are the XDAT# files for?

(08-02-2010 12:10 AM)FordGT90Concept Wrote:  Um, you gonna make an app to compile/decompile LUA files?

There are two apps that can decompile LUA 5.1: Luadec51 and ChunkSpy, but neither seem to work with Tropico's compiled LUA files. It could be the double vs integer number format or something else non-standard in the header. I certainly know enough about how LUA files are compiled now to make a decompiler just for Tropico 3, but I'll need a lot more motivation first.

As for a compiler, I just made one. (Okay, I only changed one line in the existing compiler source, but still.)

(08-02-2010 12:10 AM)FordGT90Concept Wrote:  Also, what are the XDAT# files for?

The original Tropico stored music in a bunch of XDAT##.xdt files, so that's just to simplify the process for people porting T1 music. Those lines can be replaced and deleted with whatever you want.

6-bits, seriously? That's gonna be a PITA.

Do you have a list of all opcodes?

Yes, the opcodes are 6 bit and makes the original HEX bit code hard to read straight off since they are in 8 bit. Wow, you guys are good to spot that so obvious but not.

Here is a list of opcodes I got off from reading about LUA format....

http://luaforge.net/docman/view.php/83/9...ctions.pdf

Oh, that integer thing instead of a real number used in this LUA version is a good catch. I could never figure it out.

0000 ** source chunk: simple.lua
** global header start **
0000 1B4C7561 header signature: "\27Lua"
0004 51 version (major:minor hex digits)
0005 00 format (0=official)
0006 01 endianness (1=little endian)
0007 04 size of int (bytes)
0008 04 size of size_t (bytes)
0009 04 size of Instruction (bytes)
000A 08 size of number (bytes)-------> Topico 3 uses 04 as size.
000B 00 integral (1=integral) --------> Tropico 3 uses 01 as integer.
* number type: double
* x86 standard (32-bit, little endian, doubles)
** global header end **
---------------------------------------------
OPCODES LUA 5.1

Opcode Name Description
0 MOVE Copy a value between registers
1 LOADK Load a constant into a register
2 LOADBOOL Load a boolean into a register
3 LOADNIL Load nil values into a range of registers
4 GETUPVAL Read an upvalue into a register
5 GETGLOBAL Read a global variable into a register
6 GETTABLE Read a table element into a register
7 SETGLOBAL Write a register value into a global variable
8 SETUPVAL Write a register value into an upvalue
9 SETTABLE Write a register value into a table element
10 NEWTABLE Create a new table
11 SELF Prepare an object method for calling
12 ADD Addition operator
13 SUB Subtraction operator
14 MUL Multiplication operator
15 DIV Division operator
16 MOD Modulus (remainder) operator
17 POW Exponentiation operator
18 UNM Unary minus operator
19 NOT Logical NOT operator
20 LEN Length operator
21 CONCAT Concatenate a range of registers
22 JMP Unconditional jump
23 EQ Equality test
24 LT Less than test
25 LE Less than or equal to test
26 TEST Boolean test, with conditional jump
27 TESTSET Boolean test, with conditional jump and assignment
28 CALL Call a closure
29 TAILCALL Perform a tail call
30 RETURN Return from function call
31 FORLOOP Iterate a numeric for loop
32 FORPREP Initialization for a numeric for loop
33 TFORLOOP Iterate a generic for loop
34 SETLIST Set a range of array elements for a table
35 CLOSE Close a range of locals being used as upvalues
36 CLOSURE Create a closure of a function prototype
37 VARARG Assign vararg function arguments to registers

Hey,

I got Tropico 3 on steam a few weeks ago and just bought a video card last week so I can play it, but I am definitely missing the Tropico 1 songs. I have a few questions if you don't mind helping.

1. I have two data.hpk's
....\Steam\steamapps\common\tropico 3\Packs\boot\persist\data.hpk
.....\Steam\steamapps\common\tropico 3\Packs\Patches\Boot\data.hpk

I assume the one I am trying to modify is the Patches\Boot\data.hpk?

2. I used the hpk extractor on the music.hpk and found all the music. I assume I need to put the music I wanted to add in here and archive it and put it back in the folder where I found it?

3. I found the .lua file but how exactly should I go about modifying and decompiling. Can I just edit slserpent's .lua file for the songs I am going to add (e.g. change title name), use slserpent's comiplier, replace the old one, and archive it back, and replace the old data.hpk. Or is there more stuff in the playlists.lua that I need and doing this will do bad bad things?

Thanks for the help. Sorry for asking so many questions I am going to go ahead and try this. Thanks again I love this game hopefully it will get better. I swear if I have to hear that one song with the screaming lady.....

Edit: here is an .xdt ripper http://www.the-nextlevel.com/tropico/caf...pic=9183.0

I don't understand why people don't load up winamp, turn the music down in tropico 3, and listen to their own music. For my time and enjoyment it beats decompiling and fighting with it.

I probably should do that but I don't know something about that I never liked. Also running less programs at a time would be better.

(14-02-2010 05:16 PM)JohnCub Wrote:  I don't understand why people don't load up winamp, turn the music down in tropico 3, and listen to their own music. For my time and enjoyment it beats decompiling and fighting with it.

When I started this thread, all I wanted to do was to remove a couple of the more annoying tunes, I don't generally want to swap the music out for something from my own collection. In fact I'm one of those rare people I guess that doesn't like to play my own music when I'm playing a game, doing so doesn't let me concentrate where as the music in many games is so generic that it allows one to concentrate, however there are a few really annoying songs to me in Tropico 3 that after I've heard it for the umpteenth time, I think I'd rather not hear it again the rest of my life

(14-02-2010 05:09 PM)thelife404 Wrote:  I swear if I have to hear that one song with the screaming lady.....

lol I don't understand how that song even made the cut, it's... obnoxious to me, and the worst part is that there's absolutely nothing we can do about it, short of actually shutting it off, unless you guys can get the lua's modded. the rest of the tracks are great, on the level of what we had for t1 imo. what it lacks in is variety, where t1 also had more mellow tracks to go with the upbeat ones, a real complete package.

I don't know enough to help out but I wish you good luck, you would be doing us a great service just by providing an edited playist minus that one song, it's really getting to me. even more awsome would be a "throwback mod" where we could drop in the songs from the old t1 in addition to the new tracks.

(14-02-2010 05:16 PM)JohnCub Wrote:  I don't understand why people don't load up winamp, turn the music down in tropico 3, and listen to their own music. For my time and enjoyment it beats decompiling and fighting with it.

well for those who know how, it's more like "because we can", and also you would keep the rest of the game features like auto fade for the radio guy intact. however annoying he is, and the improper greater than/less than value detection (he incorrectly warns you of events in both directions most of the time), it's still useful to let you know what's going on in the game.

Well I figured out how to replace songs so i just replaced the new tropico songs with the old. Unfortunately you can only have 15 songs and I have yet to figure out how to add more.

(14-02-2010 11:21 PM)thelife404 Wrote:  Well I figured out how to replace songs so i just replaced the new tropico songs with the old. Unfortunately you can only have 15 songs and I have yet to figure out how to add more.

would you be so kind as to post a brief tutorial, I would love to be able to mix and match, even with a 15 song limit.

(14-02-2010 05:09 PM)thelife404 Wrote:  1. I have two data.hpk's
....\Steam\steamapps\common\tropico 3\Packs\boot\persist\data.hpk
.....\Steam\steamapps\common\tropico 3\Packs\Patches\Boot\data.hpk

I assume the one I am trying to modify is the Patches\Boot\data.hpk?

You don't need to extract the Playlists.lua file to mod it. You only have to compile the script that I posted after making your own modifications.

(14-02-2010 05:09 PM)thelife404 Wrote:  2. I used the hpk extractor on the music.hpk and found all the music. I assume I need to put the music I wanted to add in here and archive it and put it back in the folder where I found it?

There's no need to re-archive the music after you make changes. The game just has to be able to find it in your Playlists script.

(14-02-2010 05:09 PM)thelife404 Wrote:  3. I found the .lua file but how exactly should I go about modifying and decompiling. Can I just edit slserpent's .lua file for the songs I am going to add (e.g. change title name), use slserpent's comiplier, replace the old one, and archive it back, and replace the old data.hpk. Or is there more stuff in the playlists.lua that I need and doing this will do bad bad things?

Again, no need to archive the modded playlists file either, just make sure it's in the right place (i.e. \tropico 3\Data\Sounds\Playlists) and compiled. If you screw anything up, you just won't hear music. To undo the mod, just delete the playlists.lua file.

(14-02-2010 05:09 PM)thelife404 Wrote:  I swear if I have to hear that one song with the screaming lady.....

Exactly why I went about doing this mod. IIRC, it's Track5.ogg, BTW.

And for those that would rather not fool with modding the music and just use another music player, let me just remind you that the in-game music will automatically fade out whenever the radio announcer or speeches occur.

---

(14-02-2010 11:25 PM)whybuybeta Wrote:  would you be so kind as to post a brief tutorial, I would love to be able to mix and match, even with a 15 song limit.

(05-02-2010 12:41 AM)slserpent Wrote:  but to replace the music files, just put an OGG format audio file into "tropico 3\Music" and rename it to TrackN.ogg where N is a number 1 to 15. this will override the music files in the Music.hpk archive. Simple if you already have some OGGs. if you need to create OGGs from MP3s, try LAME plus LameDropXPd or Razorlame to get the MP3 to WAV, then OGGdropXPd to get the WAV to OGG. if you want to know what music files to replace, you'll have to give the extracted original files a listen. get one of the HPK extractors floating around and extract "tropico 3\Packs\boot\Music.hpk".

(05-02-2010 12:41 AM)slserpent Wrote:  but to replace the music files, just put an OGG format audio file into "tropico 3\Music" and rename it to TrackN.ogg where N is a number 1 to 15. this will override the music files in the Music.hpk archive. Simple if you already have some OGGs. if you need to create OGGs from MP3s, try LAME plus LameDropXPd or Razorlame to get the MP3 to WAV, then OGGdropXPd to get the WAV to OGG. if you want to know what music files to replace, you'll have to give the extracted original files a listen. get one of the HPK extractors floating around and extract "tropico 3\Packs\boot\Music.hpk".

yeah thanks got it working, missed that part on the first page.

(14-02-2010 05:16 PM)JohnCub Wrote:  I don't understand why people don't load up winamp, turn the music down in tropico 3, and listen to their own music. For my time and enjoyment it beats decompiling and fighting with it.

(14-02-2010 11:25 PM)whybuybeta Wrote:  

(14-02-2010 11:21 PM)thelife404 Wrote:  Well I figured out how to replace songs so i just replaced the new tropico songs with the old. Unfortunately you can only have 15 songs and I have yet to figure out how to add more.

would you be so kind as to post a brief tutorial, I would love to be able to mix and match, even with a 15 song limit.

How to mix and match music in Tropico 3 (Replace not adding):
ALL CREDIT GOES TO: slserpent
Check her/his post on the first page for more info.

1. First you need to go to HPK Link and download the HPK Extractor.

2. Locate the Music.hpk file in your Tropico 3 Directory. Mine was
D:\Program Files\Steam\steamapps\common\tropico 3\Packs\boot\Music.Hpk (I have the steam version)

--Make a back up of the Music.hpk in case.

3. Use the extractor on the Music.hpk and it should extract to a Music Folder and in that folder you should see Track1-Track15.ogg Just replace what ever song you want with one of these and you are good. Make sure they are named correctly and they are .ogg *Goggle if you need to convert*

4. Archive it back up MAKE sure the directory system is the same ...extract it to make sure you have Music\*All the music files. The HPK extractor archives the current folder directory and everything else so be careful.

5. Replace your old Music.hpk in the tropico directory with the new one.


If you need to rip the old music from Tropico and have .xdt files use http://www.the-nextlevel.com/tropico/caf...pic=9183.0 (Click on the sig Viva la Tropico if the ripper link doesn't work. The instructions are a few pages down.)

Enjoy Hopefully this helps post if this is not clear.



To slserpent:
This is where I was confused I dont have a tropico\data directory. Everything is in .hpk how about you? Yea track5.ogg was the first to go ..... the horror.

(15-02-2010 12:40 AM)thelife404 Wrote:  To slserpent:
This is where I was confused I dont have a tropico\data directory. Everything is in .hpk how about you? Yea track5.ogg was the first to go ..... the horror.

yeah I was hung up on the same part, I had the repack method working too but now that I got a bit of time and caffeine to wrap my head around his notes, going this route is much better since we're not limited in the number of tracks.

*sips coffee*

my steps for those of us less familiar with the lua data structure and methodology going on here, refer to the above posts for porting/extraction/conversion to ogg (I use audacity for audio manipulation):

1. create .\music directory in your install root, this is where all your desired tracks will go, same as from music.hpk

2. create .\data\sounds\playlists directories in your install root, this is where your new playlists.lua will go.

*edit - tried running the game off extracted data only, this will not work properly. had all sorts of problems like data corruption and crashing on save, so make sure you keep all your hpk files intact, and use only the modified playlist.

3. edit/compile slserpent's script as per his directions on the first page (be careful with the array order and filenames, make sure output goes to .\gamedir\data\sounds\playlists\playlists.lua). as he said if you got typo/syntax errors or somesuch then you will just not hear any music, took me a couple tries to get it working properly with all my files as well.

thanks to slserpent et al. for doing the legwork, now I got my oldschool/newschool combo for t3 playing here.