Tuesday 27 March 2012

ME2 to ME3

As ever, it isn't just ME1 to ME2 to ME3 that aren't importing correctly. A fantastic video, made StMalk on the Bioware Social Forums, demonstrates how ME2's face codes result in something completely different in ME3.


I will say that he mentions it's fine for maleShep. I tend to disagree, following the same processes. Certainly the default male Shep hasn't had any issues I can discern. Nevertheless, this video really shows the differences between what a face code actually IS in each game.

This particular issue can be explained by the differences in slider values in ME3. Clarian, on the Bioware Social Forums, elaborates this really well. He makes the example that the minimum value for Cheek Gaunt in ME2 is substantially different to ME3. Once again, if every slider is like this, entropy rolls in and you get the effect StMalk is describing.

I think it's important to highlight this issue, since this is an example of how every importing player, ME1 or ME2 starters regardless, are affected. "Some" players don't have the issue. "All" players are affected it's just "some" are going to the forums and bringing it to the developers attention.

For the discussion on ME2 to ME3 importing see here.

Friday 23 March 2012

There and Back Again

So I started and finished ME3. I used the YAML method to generate a face code, then put that in ME2. I customized until I was happy, then used the altered face code in ME3 (after importing my save). Finally, I imported the Vector and Scalar variable collections. The result:


You can see that the in game shots are much more similar and I put up with the differences. Eventually, I came to just ignore the niggles. I played my Shep how she would be and it worked well. I will now provide my reaction, minus specific spoilers, to the endings. If you wish to stop reading, I fully respect that. Highlight the text to see it.

Now that I've done it I can see why so many people are upset about it. There was a heroism in ME2 as an ending, with moments of loss and regret, depending on your choices. ME1 took it's ending to a point where the magic of the universe endured, as it's successor.

ME3? Bioware talks about closure but I think they're missing something. The endings felt like everything that had been done was moot; you could have sat on your ass as a Spectre and still gotten one of the endings, which were hardly different from the others. Saving the base, your companions loyalty, uniting or destroying races and everything from "Well, what about Shepard?" to the final mission gate in ME3 amounted to nothing more than war assets.

The game itself is filled with heart rending moments and times where you do everything you know you needed to in the previous two games. The endings are very beautiful, especially the "best" ending, perfectly choreographed and executed, given the scenario that appears in the last 15 minutes. But that scenario, in my opinion, defeats the Reapers sure, but also everything that was unknown and untouched about the Mass Effect universe. You'll see what I mean and it is only my opinion but I now agree, it isn't the way it should have gone and I hope Bioware comes up with something more deserving of fans' continued devotion. Until then, I won't be playing again.

Wednesday 21 March 2012

Fix Update

In case you are unaware, Chris Priestly has provided an update on fixing the face import issue. In short, the team has a fix in an upcoming patch, but the patch is in certification testing.

This process, as Chris explains as well, makes sure that there are no known errors in the patch. This process is performed by Bioware, EA, Sony and Microsoft, so we will have to wait for this to occur, but I can tell you it is an important process of shipping a product or patch. I know someone will be thinking "BUT WHY DIDN'T THEY DO THAT BEFORE SHIPPING THE GAME?".

In addition, no release date for this patch is available as now, but this process can take time.

Source: http://social.bioware.com/forum/1/topic/323/index/9661093

Friday 16 March 2012

Full List of Missing Variables?

Continuing from my previous post, I mentioned that an ME3 save is missing variables that control the geometry of the face. If have since played with a manShep as well to understand this. It appears that there are three types of MorphFeatures variable:
  1. Affects male and female characters. "race_yngAsn" is an example of this. The variable is interpreted based on whether the character is male or female in different ways.
  2. Variables only for male characters. If present with females, appears to be ignored.
  3. Variables only for female characters. If present with males, appears to be ignored.
Below are the variables I believe are missing as well as possibly missing variables:
  • Ears: 
    • ears_up (possibly ears_down)
  • Eyes: 
    • eyes_bagsIn
    • eyes_bagsOut
    • eyes_BallDown, 
    • eyes_BallBack, 
    • eyes_BallFoward, 
    • eyes_BallUp, 
    • eyes_LipUpper (possibly eyes_LidLower), 
    • eyes_SlantUp, 
    • eyes_SlantDown, 
    • eyes_small (possibly eyes_big), 
    • eyes_Shape_outerPoint, 
    • eyes_Shape_squint, 
    • eyes_Shape_wide
  • Mouth: 
    • mouth_CornersUp (possibly mouth_CornersDown), 
    • mouth_LowerLipFat
    • mouth_upperLipFat
    • mouth_lowerLipUp (possibly mouth_lowerLipDown),
    • mouth_overBite (possibly mouth_underBite)
    • mouth_upperLipDown (possibly mouth_upperLipUp)
    • mouthShape_centerKleft
    • MouthShape_Diddy
    • mouthShape_underBite (possibly mouthShape_overBite or related to mouth_overBite)
    • mouthShape_thin
    • mouthShape_think
    • mouthShape_pinchedSides
    • mouthShape_Philtrum
    • mouth_lipsFat
    • mouth_lipsThin
    • mouth_cheekMass
  • Jaw:
    • Jaw_width
    • JawLower (possibly JawUpper)
  • Teeth:
    • teeth_back (possibly teeth_forward, teeth_up, teeth_down)
  • Hair:
    • HIR_BeardTipMorph
    • HIR_Beard
    • HIR_TimSelect
    • Afro
    • rollins
    • flatTop
    • flatTrop_WidowsPeak
    • HAIR_sidePart
    • HAIR_pulledBackBig
  • Other:
    • None (as in it's a variable called "None").
If you see a variable in your ME2 save that's not on this list, or is and is a possibly, let me know. Also, if there is a variable on this list that you find in your ME3 save, let me know and I'll remove it from the list as importing.

Thursday 15 March 2012

Missing Variables

Thus far, I have mentioned that the facial structure is dictated by:

  • MorphFeatures - these dictate the LODs
  • OffsetBones - Affects the animation of the face and some positioning
  • LODs - the core of the face. Inherits from MorphFeatures and will approximate the correct values when LODs are missing (rebuilt if missing).
However, yet another issue arises. Many people have mentioned that features from the previous creator were missing. I personally noticed hair styles. Beyond that, I wasn't too aware of actual missing variables. However, I eventually noticed quite a few missing from the MorphFeatures collection, and since LODs uses that to build the face, I decided to try adding missing MorphFeatures that were in ME2 into and ME3 save.


And so the plot continues to thicken. This was done with a save that had no LODs, forcing the game to rebuild them automatically. The left picture had MorphFeatures (not including the missing ones), OffsetBones, Textures, Scalars and Colours from an ME2 save. The right had the same but adding the below variables into MorphFeatures:

  • eyes_BallUp
  • eyes_small
  • mouth_CornersUp
  • mouth_LowerLipFat
  • mouth_lowerLipUp
  • eyes_BallForward
  • eyes_SlantUp
  • eyes_bagsIn
  • mouth_overBite
  • mouth_upperLipDown
  • ears_up
  • eyes_LidUpper
  • teeth_Back
  • None
  • HAIR_sidePart
  • HAIR_pulledBackBig
While I'm not certain of which, I know manShep is also missing variables. You can see the differences. Once again, nuances are being lost...

Face Codes (and why whatever you put, it won't be what you're after)

Picture by MrNiceGuy976 (http://social.bioware.com/1073264/)

In Mass Effect 2, Bioware created the face code, a really awesome way to create a Shepard on any platform playing the game. The face code provides a string of characters, separated by '.' that set slider positions for the character creator. It's a great system and works really well.

The way this works is thus: your Shep's face has a series of values setting where parts of their face are. Consider the mouth. It has several values, including the left and right edges, top and bottom of both lips, position relative to the face, as well as set positions for when it is closed or a particular sound is being said. Now, in the character creator (even Mass Effect 1), the sliders adjusted these values. "Mouth Shape" sets all those values to preset locations set in the game. You can see these values in your save, using Gibbed's save editor.

All the Mass Effect games (and most character creators actually), use slider positions as preset locations then interpreted that data into what you see. "Mouth Shape 5" isn't saved, a myriad of values associated with that shape are saved.

Now, this is why in Mass Effect 1 compared to Mass Effect 2, having the same slider positions in both character creators had slightly different outcomes; the slider positions were associated with different presets. For example, Mouth Shape 5 gave the top lip a position of <6, 4, 160> in Mass Effect 1. In Mass Effect 2, the value of Mouth Shape 5 could have changed to <6.5, 4.1, 161>. In terms of geometry, that is a minor change, but it is noticeable. To counteract this, when you imported your Shepard from Mass Effect 1, that <6, 4, 160> for your mouth was transferred, NOT "Mouth Shape 5", and you weren't able to change the look of your Shepard to protect that data.

In Mass Effect 3, the same thing has happened. Consider the continued example:
  • Mass Effect 1's "Mouth Shape 5" gives the top lip a value of  <6, 4, 160>.
  • Mass Effect 2's "Mouth Shape 5" gives the top lip a value of  <6.5, 4.1, 161>.
  • Mass Effect 3's "Mouth Shape 5" gives the top lip a value of  <6.3, 4.2, 160.75>.
Now, remember the face code sets the slider position. The segment in the face code that sets the mouth shape will be set to "5". But "5" means something different in each game. That's why facial structure looks so different using the YAML method; slider positions mean different things.

To extrapolate this further, essentially, Mass Effect 1's slider positions sometimes sit BETWEEN slider positions in ME2 and ME3. This wasn't a problem in ME2, because the raw data was imported. Why is this a problem in ME3. Can you guess? Because the raw data is not being imported...

Wednesday 14 March 2012

JKThunder's "How to Rebuild and Fix your Shepard in ME3 - PC and Xbox 360" thread

As a minor update, jkthunder on the Bioware Social forums has compiled a thread of useful information. It contains guides on how to use Gibbed's save editor as well as resources to hair meshes and other good stuff. Handy.

http://social.bioware.com/forum/1/topic/323/index/9938671

Copying LODs from ME2 to ME3 (or Warp Face)

In my previous post I highlighted how removing the LODs and forcing a rebuild cause significant differences in the face of your Shepard. I also mentioned attempting to automate the process of copying the ME2 LODs to ME3.



As you can see, I think it's been a real success. I think the character of my Shepard really shines through.

Seriously, I think this illustrates that the ME3 head morph is actually more complex than it's previous iterations. The data is there and I think the internal knowledge that Bioware has could easily transfer the nuances that lie somewhere in this collection.

Deleting LODs

So I mentioned in the first Face Effect tutorial that I didn't quite understand what the LODx collections do but it likely had something to do with the geometry of your face as well.

Well, first, LOD1-LOD3 generally don't have anything in, but maybe they do for some of you. I only had data in LOD0 and so, for interest's sake I deleted the data in this collection. What happens?


I was expecting a crash. Instead I got this. It's an interesting twist because if one saves the game, then examines this latest save you will find this data is rebuilt. Not exactly the same either, but the same number of entries (more than ME2).

This suggests that the key differences between a YAML (or indeed any face code generated Shep) and those sitting somewhere inbetween (ME1 imports, ME2 new game+ imports) are dictated by subtle nuances in this LOD collection.

Now, previously I demonstrated copying over about 100 variables in total. The LOD0 collection by itself in ME2 is 2232 (0-2231). ME3 has 2390 (0-2389). I attempted to get a macro going to copy this but unfortunately I didn't have much luck with the automation. Regardless, this might be the secret to why your Shep isn't looking the same, along with issues in the MorphFeatures and OffsetBones collections.

In fact, this effect can be noticed further by deleting the MorphFeatures and OffsetBones collections as well.

Tuesday 13 March 2012

The YAML Method

The YAML method employs pulling raw data on your character's face from the save, then using a mathematical calculation to get the closest face code in ME2 (not ME3). You can see a fantastic explanation of it here:
http://masseffect.livejournal.com/656484.html

I will not personally go through the YAML method that Bioware has suggested, since instructions have already been provided that are more succinct than I could produce. Janus Propsero on the Bioware Social forums has provided an excellent guide for generating a face code:
http://social.bioware.com/forums/forum/1/topic/323/index/9247811/3#9249327

Why does this method not always return the exact Shep you had? Mass Effect Tools and the code generation software used in this method are not erroneous at all. They're marvellous and do everything right from what I can tell. I envy the developer's ability.

From what I can tell, based on looking at the math, the reason a face code Shep won't replicate your imported Shep is because a face code is imprecise. A face code sets the positions (or ticks) in ME2's character creator. Slider positions in ME1 generated values often between slider positions in ME2. I could be wrong, but that's the impression I have been given looking at the data.

Nevertheless, this is a great method and the way I see it, your mileage varies as far as you go with it. There is no reason why you couldn't use this method, then use the Gibbed method (see the Face Effect tutorial below) to deal with colours and some minor positions.

Face Effect - Tutorial for importing variables from ME2 to ME3 (Version 1)

The Face Effect
What does this method do? It provides a lengthy movement of data using Gibbed's Mass Effect editors to provide a closer approximation of your original Shepard than the YAML method. I've only done this with femShep, but I'm sure the process can be used with manShep.

It is worth mentioning that the closer you can get your Shepard in the ME3 character creator when you import using this method, the better results you have and certainly combining this with the YAML method might be prudent. This method works by changing variables tiny amounts. The rule of entropy is partly to blame for this error and only your eye will verify how much it looks like your Shepard.

For an example (note this example didn't set the hair and eyebrows because I was lazy, the tutorial explains this too):

Disclaimer
This is a temporary fix for the ME3 import bug. The fix is produced by a fan and so I don't accept any liability to loss of data or damage to hardware. The chances of this are remote but regardless, you choose to follow this
tutorial at your own discretion. In addition, back up your save files and profile files on ME2 and ME3, just in case.

Xbox saves
If you're a PC user, skip this section. If you're an Xbox user, Dimensio kindly found a link to get your save off of and back to the Xbox.
http://www.gamefaqs.com/boards/995452-mass-effect-3/62162976

Preparation
This method requires that you already have a save file. It expects that you import one from ME2 (and thus all your plot variables as well). It also requires that you watch the first cut-scene in order to have a controllable Shepard (this isn't entirely true, but it is easier to notice differences when you can control them).

Also, you will need two versions of Gibbed's Mass Effect save editor. One for ME2, one for ME3. Links below:
Mass Effect 2 Editor:
http://mod.gib.me/masseffect2/saveedit_rev25.zip  (official version, doesn't work for Xbox saves)
http://social.bioware.com/project/4373/&v=discussions#files  (modified version, works for Xbox saves)

Mass Effect 3 Editor:
http://svn.gib.me/builds/masseffect3/ 

Extract these in folders you can find. I personally used the desktop. Both applications have a pink square as an icon, if you're confused as to what you need to see. Finally, make sure to have your ME2 save on hand so you can import it into ME3.
  1. Start ME3, go to new game then click “Import ME2 Character”. You will be presented with something that isn't your Shepard and possibly a message saying “Face reconstruction required” click the box to get rid of it.
  2. Now choose the complexion, hair style and brow style that best represent your Shepard. This is important because changing it can be more difficult and requires knowledge I don't have right now. You may also want to add scars. I know they should have imported from ME2, but I'm not certain of this (I always used the regenerator). Everything else will be set later.
  3. Once the complexion, hair, eyebrow and scars are correct (or as close as you can get) accept and start the game. You can make adjustments to other things as well, but these may be reset anyway. This method gets you closer to the Shepard you had but a lot of the leg work can be done in the character creator as well.
  4. Play until you gain control of Shepard (don't worry, no spoilers :D)
  5. Save the game TWICE, as in, in two different slots. You'll end up with Save 0001 and Save 0002, hopefully. This is so you have a backup.
Variable Import Explanation 
This section is where the magic happens and is also very laborious. You'll be double clicking, pressing CTRL+C, double clicking then pressing CTRL+V a lot, so I have done this in stages and explained what you are changing as we go. We aren't copying all the data, only the bits I understand to get a result I think is not bad.
  1. Open Gibbed's ME2 save file editor and open the save with Shepard you want the appearance from.
  2. Navigate to the “Raw” tab, then scroll down until you find “Player”. Click the + next to it to open it up, then do the same for “Appearance” then “Morph Head”. You will have a set of collections ranging from “Accessory Meshes” to “Vector Parameters”.
  3. Open Gibbed's ME3 save file editor and open the save you just started ME3 with. Navigate to the “Raw” tab, then scroll down until you find “Player”. Click the arrow next to it to open it up, then do the same for “Appearance” then
  4. “Morph Head”. You will have a set of collections ranging from “Accessory Meshes” to “Vector Parameters”, like below:


So you'll notice these look rather similar huh? That's because the system governing a face morph is very similar, minus a view variables. To explain what you're looking at, these fields you won't be using:
  • Accessory Meshes are things like hoods and visors. For Shepards, these are governed by a separate system (it's more for companions).
  • Hair Mesh is the mesh of your hair. The data input here is what's called a constant because each mesh has a name. To this end, to get the hairs available to you, you needed to set this in ME3.
  • LODxVertices are “level of detail” vertices. I will be honest in not being sure how much these affect the face but from other games these variables may only be used for long draw distances. Nevertheless, the data isn't directly transferable without some frame of reference.
These fields you might be using:
  • MorphFeatures is something I'm not sure about. When I transferred these variables, it didn't seem to have any effect. You might try not bothering with them.
  • OffsetBones is the bones under the skin that control the FACE SHAPE. This is the most important bit really, but colour also is a factor.
  • ScalarParameters is effects on the face, stuff like lighting interaction (such as how glossy the hair is).
  • TextureParameters is a collection of data just like Hair Mesh. It uses named variables which hopefully you set in ME3. This would be things like your complexion, hair and brows styles.
  • VectorParameters controls colour, including skin tone, eye colour and makeup.
Now you know what everything does. Now onto the actual movement.

Import
I'll use VectorParameters as an example, but the principle applies to all the fields above. Remember the important ones are OffsetBones, ScalarParameters and VectorParameters.

First, open the VectorParameters collection by clicking the “..” button next to it in both editors. You'll end up like this.


This is the raw data controlling your face. For example EYE_IRIS_COLOUR_VECTOR controls the colour of Shep's iris. You see a list of variables on the left section and their values (such as RGB or XYZ) on the right.

If you put the two windows side by side, you might notice some differences! That's what you want to import. Highlight the variable, say Eye_Iris_Colour_Vector in both editors and open it up on the right. Double click on the first values (in this case A, which stands for Alpha) in the ME2 editor to select it all, press CTRL and C to copy it, double click on the same variable in the ME3 editor and copy it into the respective place (again, A). The diagram below might explain it better.


Good job, that's one variable. Now you might notice that the order of the ME2 editor is a bit random whereas the ME3 editor is ordered nicely. The process I work with is to pick the variable in the ME3 editor, find it in the ME2 editor then copy it over. A note, if you find that there is a variable in ME2 but not in ME3, or vice versa, don't worry too much about it; the unfortunate thing is this isn't a precise movement.

Now, do this for each variable in OffsetBones, ScalarParameters and VectorParameters. It can take up to an hour.

If anyone needs an explanation of what they're doing or some help, just let me know.

Credits

  • Gibbed - Most of the legwork for this method comes from Gibbed's save editors and so he should get the most credit for this; being able to interoperate and import data from saves when they're compiled must have taken ages to understand.
  • Clarian - A user who followed what I was suggesting on the Bioware social forums and helped delve into things I didn't understand.
  • Malcroix - A user on the Bioware social forums who's given this method credence by beating me to the jump and posting the full idea first!
  • Dimensio - A user on the Bioware social forums who very quickly provided a link to getting saves from the X360, a console I personally don't own.
Original link: http://social.bioware.com/forum/1/topic/323/index/9661093/126#9827561

Hex Editing, this time of face structure

If you're just joining, I've managed to prove that it's entirely possible to have the same colours and lighting interactions of your ME2 Shep hair, brow, skin, teeth, make-up and eyes. This isn't a face code fix like the YAML export method (in the first post), it's a hard copy of the values from ME2 saves to ME3 saves and a long process. This demonstrates that certainly some data can definitely be imported into ME3 if it was working. You can see the results here: http://social.bioware.com/forum/1/topic/323/index/9661093/114#9780245
(or on this blog: http://faceeffect.blogspot.com/2012/03/continued-hex-editing-now-of-scalars.html)

Why did I do this? Because I wanted to know if the system used to make a face (not the character creator, I mean the raw data) was the same. If it's the same it means that data can be used and the possibility of an exact import (not the YAML workaround) exists. So far, I am led to believe it does.

This time I used the same approach with head positions. It seems copying them in this way doesn't generate the same results and tends to make the game crash. I wouldn't recommend it to anyone.

That said, perhaps that is a reason. ME2's save file is a mess when it comes to appearance if you take the time to read it. Positions are all over the place and embedded in strange places in the save file. They're also usually fractionally different to very similar looking ME3 ones. ME3 puts all the eye positions together, then the mouth, then the jaw and so on,
just as you see it in the character creator.

Perhaps the reason is that ME3 expects an order and ME2 supplies 75% of that?

Anyway, I've hit a wall with this and since we'll likely see a fix (or at least an announcement) this week, I'll retire from working on it.

Clarian said:
"I've been trying to follow Pheonix's lead and examining the save file in a hex editor.  I think my biggest problem is the facial structure, since the whole shape of my Shep's head is just off. I'm guessing that the things in the save file that correspond to 'facial structure' are the 'races' (race_yngAsn, race_oldBlk, etc.) 
It works rather differently than eye color, though.  With eye color, there's one field (vector_eyecolor, or whatever it was), and apparently the value in that field defines the color of the eye. But with 'race', there's a different field for each one, and either it's blank or you have a value in it.  But I don't think that one value could be enough to define the entire morph.  So...it seems like blank vs. populated just tells it which one you're using, which points somewhere else to the full definition. 
The thing is, with eye colors, some values weren't available in the cc.  So, put in a value that the cc didn't let you have, and now you can have it.  But with races, the list from my ME2 save and ME3 save are the same.  Unlike the purple eye color, 'race_oldAsn' is still there in both of them.  But it just translates into something quite a bit different in 2 vs. 3. 
Now, the gibbed ME2 save editor has an 'offset bones' collection with all the facial features, cheekbones, lips, etc.  But one, there's no ME3 save editor yet (is there?), and two, I bet it would be really hard to edit those without being able to see the results of what you're doing real-time. 
So...thus far, I've failed to figure out how to fix facial structure. But still, what Pheonix found is great for all the color palettes, and a big improvement over the ME3 default."

From what I can tell, and I think you're on the right track, the "race_oldAsn" and so on refer to the preset head structures. These were in ME1 as well, I think under the name "Facial Structure". This likely either sets the bones to be used or sets them in a particular place. Certainly the variables are different, possibly because of an order of magnitude (i.e. ME2 had small offsets, ME3 uses bigger ones).

And yeah, so far all I've managed to show is that you can get yourself crazy eye colours like bright pink in ME3 :D. That said, I think if we don't get an import, changing the colours really helped make it look like the Shep I had. Here's hoping for a save editor soon.

Original link: http://social.bioware.com/forum/1/topic/323/index/9661093/116#9787351

Continued Hex Editing, now of Scalars

Clarian provided some useful input...

Might be worth a look. I certainly haven't figured out face structure yet but I know the variables are there.

As an update, I just did scalar vectors as well. These are important because they control how light reacts with surfaces, especially hair:


As you can see, this makes a wealth of difference. In total the makeup, eyes, teeth, hair colour, brow colour and skin tone all changed. What this demonstrates is that it is entirely possible to bring in appearances from your ME2 save.

Original link: http://social.bioware.com/forum/1/topic/323/index/9661093/114#9780245

Hex Editing Colours

I have something to share with you all. Remember how I said colour values where held in vectors in your save files? I decided to mod my ME3 save a little such that these colour values were the same as those in my ME2 save, with a little help from Visual Studio 2008 and ME2CRC. This was done by hand, hex editing, not using any save editors.


Left is original that ME3 CC set up. Right is my modded version using ME1 colour pallete. Notice how the face code is the same. These are the colour values from ME1, not ME2. I'm going to try experimenting with some of the other values soon.

Original link: http://social.bioware.com/forum/1/topic/323/index/9661093/113#9779363

Set Up

I have been requested over on the Bioware social forums to provide a blog for my attempts to fix the face import errors in Mass Effect 3. Following are posts regarding this.