Conquest of Elysium 4

Conquest of Elysium 4

Marlin 1 May, 2016 @ 4:17pm
Extracted from coe4.exe: Class and Recruitment data
Update February 2018: The download link, below, is now to new data fresh from CoE 4.26!

Like before, the format is c4m files (plain text files) with pseudo-mod code. And, for the best reading experience, I recommend a text editor with CoE syntax highlighting, such as the schemes I have made for Gedit and for Notepad++ – both of them incidentally now updated for v4.25 – or, if nothing else, the syntax highlighting of just a Unix shell script. (The last one will at least make comments and string literals stand out from the rest.)

Whatever you do, I would advise you don't use the Notepad that comes with Windows. (If you're on Windows, you'll be much better off with Notepad++[notepad-plus-plus.org].) Microsoft Notepad and CoE modding (or just about any serious text processing) are two things that simply don't belong in the same sentence.


The rest of this post is what I originally wrote in May 2016:
¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯¯
So...

I finally got around to take a good look at what the class and recruitment data in the executable actually look like, and found they weren’t at all as horribly indecipherable as I had feared. In particular, data seem to be generally stored in fixed-size blocks, thankfully also often begun with some telltale plain text string. Which greatly facilitates comparison between blocks, in order to figure out what each byte and bit might represent. Numbers seem to be mostly 4-byte (signed) integers, each beginning at an offset evenly divisible by 4. (When using a hex editor, I find it helpful to have it display bytes in groups of 4.) Although there are a few 2-byte integers too.

The ideal, in my view, would have been human-readable text files (e.g. like mod files in format) with the definitions of all the stuff in the game, classes, units etc. In their absence, though, I believe things could have been worse than they are.
________________________________________

I was eventually even able to put together some Python scripts to extract the data into... Not meant to be used as actual mods, of course. (There aren’t mod commands for all the data in the executable, and I don’t understand all of it anyway; see the comments.)

I could now kick myself for not doing this earlier. I did in the past actually waste some considerable time trying to establish only some of this through arduous tests.

For the best reading experience, I would recommend a text editor with CoE syntax highlighting, such as the schemes I have made for Gedit and for Notepad++, or if nothing else the syntax highlighting of just a Unix shell script (will at least make comments and string literals stand out from the rest).
________________________________________

Notes:
  • The file on standard recruitments (standard troops, standard wizards, paid heroes and assassins) is not extracted from the executable, as I cannot there identify the data for it. It contains guesstimates based mostly just on tests that I also now got around to perform. I now feel fairly confident this is how it really works, though there is almost no way to be entirely sure.

    (Again, I wish I had done this earlier. There have been times playing this game when the capturing of extra libraries felt almost like a pointless ritual based purely on superstition, as I had so little an idea what the actual probability bonuses were.)

  • The other files contain data actually found in the executable – complete with any errors there might be. Not that I expect there to be many errors, of course, but, over in the CoE3 forum, user Ridiculus recently discovered what seems to be a corruption of recruitment data for the CoE3 witch (which must have happened in a patch to CoE3).
________________________________________

For the curious, interested in a comparison with CoE3, I posted the corresponding CoE3 data in the CoE3 forum.
Last edited by Marlin; 14 Feb, 2018 @ 6:22am
< >
Showing 1-15 of 25 comments
Althaea 2 May, 2016 @ 3:01am 
You know, even if it's bizarre, since they're alright with things like the Dominions Mods Inspector, I'm vaguely afraid that if I click that link, Johan is going to show up at my door with an offer I can't refuse the very next day. :P

More seriously, thanks. I appreciate the insight into the guts of Elysium.
Marlin 2 May, 2016 @ 6:48am 
Gulp! 😟

I seriously hope this isn’t something Johan disapproves of. Maybe I should have asked him before posting, but the thought honestly didn’t even occur to me. Like you say, precisely this sort of thing seems to have been happening for a long time regarding Dominions without a lifted eyebrow from Illwinter. I believe I saw a thread where even Edirr participated in the deciphering of data extracted from the Dominions executable.

I guess it comes down to whether the relative lack of detailed documentation is intentional (to benefit the sense of exploration regarding just about everything in the game), or if it’s just due to Johan or other Illwinter associated people having never gotten around to doing it. (Documentation is work, after all.) In the latter case, I’m performing a documentation service here. In the former case, I have just committed a heinous crime (morally).

Speaking for myself, I don’t see the fun in “exploring probabilities” – at all. Randomness makes it very hard to tell when you are done “exploring” and what it really was you “discovered”. So at least as far as the chances of recruitment offers go, I have a hard time believing the lack of documentation is really intentional. But maybe I am wrong?
Althaea 2 May, 2016 @ 8:59am 
To a degree, I think. The emphasis (especially in Conquest of Elysium as compared to Dominions) is perhaps more on not knowing exactly how a given action will turn out; you are supposed to judge risk and reward, without being able to eliminate the risk as you can in many other games.

The process of figuring out just how things work is part of the game, I think; and it's up to players whether they are willing to share their discoveries or not. A lot of the great resources Illwinter's players take for granted weren't create by Illwinter themselves, after all. That this both saves them from having to do extra work that they don't like doing and that it reinforces the themes they like in their games are probably both significant factors. :P

While I cannot say that I know their minds, I think they don't mind players using such resources or creating them, especially since it doesn't actually cheapen the gameplay. The mod inspector for Dominions won't help you win that game, and neither will this help with winning in Conquest of Elysium. And absent of context in the form of experience with the game, you wouldn't even realize what all these numbers imply or mean to begin with.

So, all jokes aside, I don't think it'll be a problem. :)
Marlin 2 May, 2016 @ 6:01pm 
Originally posted by alguLoD:
you are supposed to judge risk and reward, without being able to eliminate the risk as you can in many other games.
Just to avoid any misunderstandings, I do love dice and randomness in games in general, and in this game certainly. Risk management is a skill (often poorly understood by dice haters) – but in order to judge risk and reward, you obviously need to have a decent idea of the probabilities involved.

For that matter, even disregarding any considerations of risk management, I can also appreciate the adventure aspect of a game where I really don’t know anything at all. Moving out into the unknown, only to perhaps be stomped by a dragon, or something, the very first turn, can be kind of hilarious.

Where things become unsatisfactory to me is when I have seen most or all possible outcomes, understood everything - except, only, their probabilities. (Establishing probabilities through tests is laborious and not fun, in my view, and a game is supposed to be fun.) Ideally, if probabilities are deliberately kept hidden initially, for the adventure or exploration aspect, I would like them to be revealed after I have tried them.

But the topic isn’t trivial and others will no doubt disagree, perhaps Illwinter too.
Last edited by Marlin; 3 May, 2016 @ 1:46am
Orpheus 3 May, 2016 @ 5:03am 
Thanks for this Marlin, I can see this being a really useful resource both for myself and many other players (and modders!)
carn112004 4 May, 2016 @ 7:44pm 
Thanks for good work

Any idea how the additional resource cost can be modded?

e.g.
addunitrec "Androphag Cavalry" 100 4 50 0 5 # +fungus: 25

what the mod command for "fungus 25" would be.
Marlin 5 May, 2016 @ 6:32am 
Sorry, no. I don’t think there is any mod command for that. I haven’t seen one.

I tried to make the script express extracted data in the form of real, working mod commands as far as I could. So, for instance, two of the recruitment bitflags are shown as the commands templerec or libraryrec when set. Had I known of a command for extra recruitment costs, I would have similarly used that. Since I don’t, I just have the script write those costs in comments.

I also don’t know of a mod command to make extra recruitment offers available in locations of certain terrain types, so the script merely writes those terrain types in comments too.
________________________________________

Obviously, any further ideas on this topic – what the unknown class abilities or bitflags might be, or if somebody even knows of a mod command that I am missing – are more than welcome. 😉
Sorry for the lateness of my response here; I've been moving house and it's been a big job.

Marlin, thank you so much for this! You've saved me dozens of hours of tediously recording every class's recruitment offers, and given me more accurate data than I would have gotten from that anyway.

I completely agree with you about wanting to know the probabilities of random events, including recruitment offers, to such an extent that, shortly before moving (so back during v4.16, on the off chance anything related has changed in v4.17), I did a test in which I recorded every recruitment offer I received as the Baron (with no temples or libraries yet, to avoid their complications) for 300 turns. Most of my data from that test matches well with your data; most of the discrepancies are close enough that they can be reasonably explained by my relatively small number of data points.

However, these five recruitment offers for the Baron from your data:

# % # gold die iron addunitrec "Archer" 5 10 70 0 0 addunitrec "Crossbowman" 5 10 70 0 7 addunitrec "Spearman" 5 10 70 0 0 addunitrec "Swordsman" 5 10 70 0 7 addunitrec "Trebuchet" 10 2 75 0 75

are very interesting. The Archers showed up about as often as they should (I got 6.67%, but that's close enough), but none of the others was ever offered. With a 5% chance per turn for 300 turns, that's about a 1 in 5 million chance for each of Crossbowmen, Spearmen, and Swordsmen to never be offered. For the Trebuchets, with a 10% chance per turn, it's 1 in 53 trillion.

I think we can reasonably conclude that those offers aren't actually working. Interestingly, I remember those offers from playing as the Baron in CoE3, and it was my surprise that they seemed to have been removed in CoE4 that led me to start my recruitment data testing with the Baron. Since you've found that those offers are actually still in the executable, we can conclude that either the data in the executable isn't what's actually being used now (which I doubt, since it's been changed since CoE3, and why would they bother to change it if it wasn't going to be used?), or there's been a change in the engine since CoE3, an unintended consequence of which is that these entries are no longer able to be expressed in game.

That's where the Archers may be revealing. Of the 5 bulk, discount offers, the Archers offer is the only one that works. Why? What's different about that offer? The obvious answer is that the Archer is the only one of those 5 units that doesn't show up elsewhere on the Baron's recruitment list. I suspect the new rule in CoE4 is simply that any given unit type can only be placed in a given month's recruitment list once, at most.

I can think of 4 likely methods that the engine might use to decide priority when faced with multiple entries containing the same unit: first processed, last processed, first successfully triggered, and last successfully triggered.

From my data, we can reasonably exclude both "lasts", as the malfunctioning offers were lower on the list than their duplicate unit entries. If the method was "last processed", then the normal 5 Crossbowmen, 5 Spearmen, 5 Swordsman, and 1 Trebuchet offers that occur every turn wouldn't work, and they do. If the method was "last successfully triggered", then the bulk, discount offers should work at 5% and 10% as listed, though they would prevent the corresponding normal offer on any month they triggered. This doesn't happen either, so it can't be that.

That leaves "first processed" and "first successfully triggered". Unfortunately, the Baron's recruitment list doesn't allow us to distinguish between these methods, since the normal offers for each duplicate unit type are both listed first (so they'll be processed first) and have 100% chances (so they'll also always be the first to successfully trigger).

Looking through the rest of the class recruitment lists, I see two other cases of duplicate unit types. The Senator has this:

selectclass 8 # Senator # % # gold die iron addunitrec "Hastatus" 100 5 45 0 5 addunitrec "Hastatus" 5 10 65 0 5

which has the same problem--the first entry has a 100% chance, so it doesn't allow us to distinguish. However, the Dwarf Queen has this:

selectclass 20 # Dwarf Queen # % # gold die iron addmercrec "Pikeneer" 4 10 150 0 10 addmercrec "Pikeneer" 2 20 250 0 10

If the method is "first processed", then only the 10 Pikeneer offer should ever happen. If the method is "first successfully triggered", then the 20 Pikeneer offer can happen, though it will occur slightly less than the listed 2% of the time (1.92%, instead), since it won't get the opportunity to be checked on any turn in which the 10 Pikeneer offer succeeds at its trigger chance.

I thought I could remember having seen the 20 Pikeneers offer, so I ran a quick test as Dwarf Queen to see if it would happen again. As you can see here, it did:

https://steamhost.cn/steamcommunity_com/sharedfiles/filedetails/?id=696162964

To be extra sure, I also ran a test with the following modded recruitment list:

selectclass 6 # Bakemono # % # gold die iron addunitrec "Bakemono Spearman" 50 10 40 0 0 addunitrec "Bakemono Spearman" 100 5 25 0 0

If it was actually possible to have a given unit show up more than once on a given month's recruitment list, the above should have caused it to happen quickly. That didn't happen, and after the 25 turns I tested with this mod, the chance of that occurring randomly is about 1 in 33.5 million--enough to satisfy me that it wasn't possible.

The results I got were 15 offers of 10 Bakemono Spearmen and 10 offers of 5. That success rate for the offer of 10 is a bit off from the listed 50%, but certainly close enough given the paltry 25 data points. More importantly, I got one and only one of the two offers every turn (the offer of 5 did not happen the listed 100% of the time, but rather only when the offer of 10 didn't happen), and I did see both types of offers during the test. Of the 4 prioritization methods I proposed, only "first successfully triggered" could produce these results.

From all of the above, we can conclude that when multiple entries on a class recruitment list contain the same unit, the first of those entries to succeed at its chance will be the only of those entries to be offered in a given month (or else that the priority is determined by some method I didn't imagine which nevertheless produced indistinguishable results). Thus, the following offers:

selectclass 1 # Baron # % # gold die iron addunitrec "Crossbowman" 5 10 70 0 7 addunitrec "Spearman" 5 10 70 0 0 addunitrec "Swordsman" 5 10 70 0 7 addunitrec "Trebuchet" 10 2 75 0 75 selectclass 8 # Senator # % # gold die iron addunitrec "Hastatus" 5 10 65 0 5

appear to all be impossible to receive, since they are each blocked by a 100% offer of the same unit higher on the list. If true, I'd imagine this qualifies as a bug.

So, how about it? Has anyone ever gotten any of the above offers in CoE4? If not, I'll go ahead and report this in the bug thread.

Also, Marlin, I saw this statement by you in the bug thread:

Originally posted by Marlin:
I may at some point post the monster definitions I have extracted from the executable, if Johan doesn’t object to that.

I would find such a resource to be absolutely invaluable, and I'm sure I'm not the only one. If you do decide to post it, I will be very grateful!
carn112004 3 Jun, 2016 @ 1:28am 
Good work dracula.

That explains why some recuitment options i tried to add, did not work.
carn112004 3 Jun, 2016 @ 2:17am 
modding commands for resolving the issue about untriggered recruits:


newmonster "Grouped Crossbowman"
copystats "Crossbowman"

newmonster "Grouped Spearman"
copystats "Spearman"

newmonster "Grouped Swordsman"
copystats "Swordsman"

newmonster "Grouped Trebutchet"
copystats "Trebutchet"

selectclass 1

addunitrec "Grouped Crossbowman" 5 10 70 0 7
addunitrec "Grouped Spearman" 5 10 70 0 0
addunitrec "Grouped Swordsman" 5 10 70 0 7
addunitrec "Grouped Trebuchet" 10 2 75 0 75



newmonster "Grouped Hastatus"
copystats "Hastatus"


selectclass 8
addunitrec "Grouped Hastatus" 5 10 65 0 5


One can also create further units (e.g. "Massed ...") to add further recruitment offers for same unit. And the cost of Hastatus is not consistent with the costs of the baron ones, it should be 63 0 7 for same 40% over cost for 5 respective units.
Marlin 8 Jun, 2016 @ 3:33pm 
Great findings, Colonel Dracula.
Originally posted by Colonel Dracula joins the Navy:
...(so back during v4.16, on the off chance anything related has changed in v4.17)
As far as I can tell, the update brings no changes to the recruitment data.

Originally posted by Colonel Dracula joins the Navy:
I did a test in which I recorded every recruitment offer I received as the Baron (with no temples or libraries yet, to avoid their complications) for 300 turns.
I did something very similar with CoE3 some time ago, and got data fairly well matching what I have now extracted from the executable. (My guesstimate then was a 5% chance of each of the “bulk offers” in the Baron’s list, also the one of 2 trebuchets, so I was slightly off on those last ones in my test.)

I guess I got so blinded by the agreement of the extracted data with those earlier CoE3 tests that I didn’t consider any changed behavior in CoE4. But now that you mention it, I believe you are entirely right.

Not only does the CoE4 behavior differ from the CoE3 one, you have clearly managed to pinpoint exactly how.
Originally posted by Colonel Dracula joins the Navy:
when multiple entries on a class recruitment list contain the same unit, the first of those entries to succeed at its chance will be the only of those entries to be offered in a given month
Your findings seem thoroughly solid to me, and, yeah, I am pretty sure it must be unintended and a bug. So I’d say go ahead and report it.
________________________________________

There is one little thing I’d like to add: In CoE3, if a chance is set to -1 (minus one) for an offer, that offer will appear simultaneously with whatever offer is preceding it in the list. I don’t think this was ever documented, but it is how it works (also usable in CoE3 mods).

In CoE4, this doesn’t work either, but at first I was inclined to think it might have been a mechanic intentionally dropped. It’s mostly replaced by reclimiter "+something" in CoE4. (E.g. in CoE3, the “minus one mechanic” is used to have an amazon priestess appear simultaneously with a sorceress and a gang of amazon warriors, whereas in CoE4, the offers are separate but the priestess is made a requirement for the others.)

However, there is one entry still with a minus one chance in the CoE4 lists, the one of the retiarius – or pair of retiarii. In CoE3, you would, as the senator, be offered a pair of retiarii anytime there were gladiators for hire – but in CoE4, they just never show.

In light of your findings, it seems likely this one is another CoE3 mechanic that just unintentionally stopped working in CoE4.
carn112004 8 Jun, 2016 @ 10:48pm 
Originally posted by Marlin:
However, there is one entry still with a minus one chance in the CoE4 lists, the one of the retiarius – or pair of retiarii. In CoE3, you would, as the senator, be offered a pair of retiarii anytime there were gladiators for hire – but in CoE4, they just never show.

In light of your findings, it seems likely this one is another CoE3 mechanic that just unintentionally stopped working in CoE4.

Means i will include them in my mod.

Can you extract the probabilities/values for summoning chances?
Thanks to carn and Marlin for confirming the recruitment list duplicate unit issue. I'll report it in the bug thread.

Marlin, I agree that the fact that -1 chance is still used in the Senator's recruitment list but no longer works, leaving retiarii unrecruitable, is very probably a bug. Thanks for finding it, and please do report it.
johan  [developer] 20 Jun, 2016 @ 3:35am 
Good catch with the recruiment offers, it has been fixed and a new update should be available now.
Thanks so much for all the fixes in 4.18, Johan (and Kristoffer). We really appreciate it!
< >
Showing 1-15 of 25 comments
Per page: 1530 50