ECU Simulator next gen - 5AM

[Meinolf]

Hi,

it's not uncommon for programmers to leave their signatures or witty comments in the code, which is only shown when certain key-combinations are pressed.
In this instance it seems that one of the programmers working on the BIN of the 2-Lambda models hid the following message in 2 Canbus packets. "Streetfighter"

I normally look at the hexdumps in 16bit LoHi-notation, so ASCII characters are not usually visible. Just for the hell of it I scrolled through the Aprilia code in 8bit notation.

And found this message string: SONO UNA Ducati. Which translates to "I am a Ducati".

Some of the programmers are real jokers....





The string "STREETFIGHTER" is displayed on the Guzzi dash after power on, I can't remember seeing anything similar on the Mana. I'm curious to see if the string is send over the Aprilia CANBus as well.

Cheers
Meinolf

[Chuck in Indiana]

 

[Meinolf]

Hi,

it's snowing again and well below 0°C. A good opportunity to follow up some thoughts. Firstly the infamous Fuel Phase table (I'd give my left arm to learn who selected this name based on which thinking) and secondly the Ignition Dwell table. And as a plus the battery voltage legend. But first things first.

The Fuel Phase table is one of the bigger arrays in the 5AM, using 20x32 cells and having TPS and rpm as legend. The original table looks like this




Now the question was, which function could this be describing. Rather large and sudden value changes and no obvious relation between value changes and legend values. It looked strange.

For the longest time I tried to figure out a way to trigger the scope to a fixed reference. The obvious choice was the 2 teeth hole in the engine speed signal, but I haven't found out how I could get my scope, a Rigol DS1054Z, to trigger to the lag. Then the idea was born to do this in a roundabout way. Set ignition values and all relevant trim tables (temperatures, baro, idle mode, ...) to 0, then the pulse would be fixed in a relationship to TDC, regardless of changes to rpm or TPS.




Then the Fuel Phase table was filled with staggered values suitable for identifying steps and uploaded to the ECU.




Then all relevant TPS settings were selected and the time delta between the respective flanks of the injection and ignition pulse were measured. And lo and behold, a structure became visible. Low values moved injection pulse end towards TDC, large values moved the pulse end to earlier timing.

So the working hypothesis now is that Fuel Value (Duration of injection pulse) + Fuel Phase Value (timing for end of injection pulse) determine the injection event in regards to duration and timing.

More to follow

Cheers
Meinolf

[Meinolf]

Hi,

it's been a cold day with lots of new snow. I'm glad I used the better weather during the last couple of days to service my motorcycle pool and especially the Norge and Futura, which are new, and tailor-suit them to my requirements. The next days will be to cold enjoy staying in the workshop.

But, it must be an ill wind which brings no good. Time invested in the ongoing quest paid off. The Fuel Phase table is mostly revealed, some details need to be validated, that's for the coming days.

One of the results of todays work was the identication of the ignition dwell table. That's the one which governs the time the ignition coils are charged dependent on the battery voltage. The formula used is X/500 = charging time in ms.





And just as important, this validated the assumed battery legend table at 0x4D056. The formula used is X/10 = voltage. There at least two other 16x1 tables which are likely to use it. Which helps a lot in trying to identify what their function is.

From some of the previous replies I gather that this seems like mysterious stuff to many. Not so. Disassemby and programming are not my core competencies. So the tedious route is the one I'm using. 2 identified tables out of about one hundred unknown, plus another factor used in a formula found helps a lot in narrowing down the search for the remaining ones.

We've made a lot of progress since John Th. from England has joined this endeavour. He's digging into the disassembled code in a very British way. Soft-spoken but very!!! knowledgeable.

Cheers
Meinolf

[pete roper]

That is VERY interesting and might explain some of the slightly weird things I've noticed about the running of bikes with marginal batteries. Thanks.

Pete

[Meinolf]

Hi Pete,

which were these "slightly weird things" you've noticed?

Cheers
Meinolf

[Meinolf]

Hi,

I firmly believe in some fundamental truths.

Firstly, sleep on it for a night
Secondly, apply Occam's razor
Thirdly, with sharing you make friends
...
... Numerology may actually make sense

Applying all of them at the same time is a powerful tool.

So in this case. While I was researching using the ECU simulator with my hands-on approach, John Th. is busy deep-diving into the disassembled code.Some days ago he mentioned that the code is using a counter running from 0-7199 over two crankshaft revolutions. Which, as it happens, turns out to coincide with the angle of one camshaft revolution if divided by 10.

7200/10 = 2x360°

Now, throwing the number of teeth, both those actually present (46) and the ones missing (2), into the ring leads to another number.

720° / (46 + 2) = 15° crankshaft/tooth

And he mentioned that the formula applied to the Fuel Phase table is likely X/10 = ° Crankshaft BTDC.

Mixing the tidbits from different sources, sleeping it over and slowly shaving with Occam's razor in the morning after leads to some conclusions.

The Fuel Phase table contains values for the end of injection pulse, and quite likely for ignition pulse as well, in ° crankshaft. A likely algorithm for the calculation of the beginning of the respective pulses is

(Value_inj/ign * trim values) + Fuel Phase values = Start of pulses in ° BTDC

The trim values would take care of the voltage dependencies of the inductive nature of the ignition coils and injectors. And quite likely there's a table or scalar for the deadtime of the injectors as well.

The numbers 7200/720 and 15/150 also provide hints which values or value ranges to search for in the quite large number of scalars, both known and assumed ones.

Chasing scalars has proven to be quite successless. A considerable number of switching values have been determined. Such as revolutions since start as duration of the warm-up function, fan On/Off temperature, closed loop area in regards to engine temperature, TPS and rpm or rpm's at which the idle fuel goes active/inactive.

But, not many of the discrete absolute values could be connected to a scalar yet. Quite the opposite, at least for idle fuel the switching point is the notional index of the y-axis. When increasing rpm the idle fuel function stops after exceeding the last 1/16ths of the legend and kicks in when decreasing below the 2nd from top 1/16ths.

It's likely that other switching points are also linked to notional indices and not to discrete values.

Cheers
Meinolf

[pete roper]

Hi Pete,

which were these "slightly weird things" you've noticed?

Cheers
Meinolf

I'll email you in the next few days. Busy as a very busy thing right now but battery voltage seems to affect certain things.

[Meinolf]

Hi,

it's not that the work on the 5AM simulator has stopped, but since May we have terrific weather here in Germany. So plenty of driving, hiking (including a week in Cyprus), working in the garden and servicing my motorcycle fleet took up a lot of time.

So, what happened in the meantime. The code analysis of the 2V BIN continued, the results were compared to the 8V BIN and also the Aprilia Mana BIN. The XDFs for all three are now covering almost the entire code and data area, consisting of hundreds of tables, scalars and flags. I believe that most of the essential ones are understood and validated, but there are plenty of unknowns left. Copies were sent to Peter Roper, Beetle, Beard and PaulDaytona. And a Mana driver from Portugal, Luis, already used the XDF to disable Lambda and thus fix issue which have troubled him.

I also began logging data on my Norge 12002V. Initially with the Innovate LM-2 and two lambda sensors, but one channel died during the tests. I have been using the LM-2 for several years, this is the most troublesome and unreliable tool I've ever come across and Innovate's technical service is horrible. Plus, it eats WBOs. The only good part of the package is the software for analyzing the logged data. The LM-2 is now moved to the 5AM simulator, I'll explore if it's OBD communication can be hooked up to the 5AM to monitor the status messages sent over the K-line.

So the two ZT-2 were moved from the V11 to the Norge. Now the ZT-2 are very robust and reliable devices, but they suffer from a number of shortcomings in comparison to the LM-2.

First, TPS is logged as %-value. That's acceptable if the number of breakpoints is not more 16 and the delta between the breakpoints is big enough to be expressed as a percentage integer value. It worked on the 15M, even though I had to change the TPS breakpoints to accommodate the resolution. It's not good enough for the 5AM, which uses 20 TPS breakpoints, of which the first ones are very close. 4.59°, 4.7°, 5.2°. Now with a resolution of (85°-4.59°)/100=0.801° those breakpoints can't be discerned, all are in the range 0-1° and that's the most difficult area to log and analyze. And even and larger TPS values it's critical because the bi-linear extrapolation of the code interferes with associating values to a specific breakpoint.

Second, ZDL, the Zeitronix software. It displays the log, but is unusable for analyzing the logged data. And that's a real problem, as the ZT-2 logs at ~60Hz. Which amounts to >200k datasets (each containing lambda left/right, TPS, rpm, engine temp, EGT and MAP values) in one hour of driving. In the past a rather tedious route was taken to analyze the data. The log was loaded into ZDL and exported as CSV. Beard wrote a program to convert the CSV to DIF, which could be then imported into Logworks, the Innovate software, for visual inspection and analysis.
However, the main work was done with the Excel worksheet created and maintained by Punch, originally intended analyzing Innovate logs. A really great piece of Excel coding, but feeding the amount of datasets  created by ZT-2 into the worksheet eventually brought it to its knees, a re-calculation took up to 15min.

So, quickly more than 10h of road logs were logged, but no reliable method of analyzing the data was available. I mentioned this in the German Guzzi forum, where most of the discussion happens, and, not surprisingly, Beard again came to the rescue. He proposed to write a data base program which used SQL to import and analyze the data. Initially we used CSV exports from ZDL, but quickly decided to try and use the native log files without the error prone and time-wasting export procedure. Zeitronix kindly supplied the specs of the serial communication and the data structure in the log file and the work began in earnest.

The result is a very usable piece of software to analyze lambda values, rpm and TPS. TPS and rpm values are user-definable, %-ranges for TPS and rpm also. A minimum number of datasets used for filtering can be selected and also data is displayed as AFR or Lambda. Zeitronix logs can be loaded individually and can also be appended. So the >10h logs resulted in a single database of more than 2.1mio datasets. Which gives more than enough datasets at each breakpoint to arrive at meaningful data, even taking into account the effects of acceleration/deceleration and so on.

And, it's blindingly fast. Re-calc of the 2.1mio datasets takes ~4s.





This morning the analysis was brought into a new BIN. And, after one hour of logging with this BIN and looking at the resulting data the conclusion is that never before could I apply logged data so quickly, effortlessly and accurately. The after-analysis showed that the delta of the left/right cylinder lambda values are almost all <= 0.01 and in line with the set lambda target values.

Last Saturday Beard came over, he asked me to straighten the rotor from his washing machine on my lathe, and while we were turning and chatting an idea came up to circumvent the TPS %-issue of the ZT-2.

The ZT-2 has several input ports - WBO, TPS, rpm, User1/2 for 0-5V, EGT and MAP sensors. When logging 2 cylinders two ZT-2 are slaved, the 2nd lambda signal is feed into one of the 1st ZT-2 voltage input ports. The 2nd voltage port is free for other purposes, I've used it to log engine temp, air temp, oil temp or voltage.

Now the idea is to feed the voltage of the TPS into the 2nd voltage port. It logs the 5V with a resolution of 5V/256=0.0195V, or in other words, the TPS range of (85°-4.59°)/256=0.314°, an improvement of 255% compared to the %-resolution. Thus a very accurate mapping of TPS breakpoints is possible. The largest error (deviation 4.3%) would be at 2nd TPS breakpoint of 4.7°, all other deviations are in the 0.1% range. That's good enough.

So, the quest continues. And, again, kudos to Beard for his splendid support and un-ending patience with my ignorance.

Cheers
Meinolf

[beetle]

Good job. I too, found the LM-2 to be a piece of junk. Luckily, I got my money back! I've stuck with Innovate, however. I use LC-2's, SSI-4 and PL-1 for data logging.

[Meinolf]

Hi Mark,

we are continuing the work on the program. Beard wants to implement reading native Innovate logs as well. I've approached Innovate and asked for specs of the log format, but their support is spurious. Do you (or anybody else) have the specs?

Cheers
Meinolf

[beetle]

Only this:


https://www.firetooth.net/confluence/display/public/Innovate+Motorsports+D32+file+format

[Meinolf]

Hi Mark,

thanks.

https://www.firetooth.net/confluence/display/public/Innovate+Motorsports+D32+file+format

following the link and doing some further searching this also was found:
https://www.innovatemotorsports.com/support/manual/OT-2%20SDK.pdf
https://www.innovatemotorsports.com/support/downloads/Seriallog.pdf

and this seems to be the current version: https://www.innovatemotorsports.com/support/downloads/Seriallog-2.pdf

That should help us in deciphering the logged data.

Cheers
Meinolf

[Meinolf]

Hi Mark,

Good job. I too, found the LM-2 to be a piece of junk. Luckily, I got my money back! I've stuck with Innovate, however. I use LC-2's, SSI-4 and PL-1 for data logging.

I'm not familiar with the LC-2, SSI and PL-1. But, if they also create D32 log files, another tool written by Beard yesterday might of use to you. It extracts data directly from the unedited D32 file and converts it to a CSV format.

When deciphering the D32 file Beard found a discrepancy in the data created by the 4th analog channel. The conversion of the logged hex data into CSV produced different values than expected. He quickly found the solution though (he always does), every 2nd value is the average of the preceding and following value and not a logged value. Not according to the specs, I've sent an inquiry to Innovate.

AFR1    AFR2    RPM     A1        A2        A3        A4        RPM2
13.33    14.04    1140     0.52      2.74      1.48      3.07      0
13.29    14.04    1140     0.52      2.74      1.47      2.83      0
13.24    14.02    1140     0.52      2.74      1.48      2.59      0

3,07 - 2,59 = 0,48

0,48 / 2 = 0,24

3,07 - 0,24 =  2,83

Cheers
Meinolf

[beetle]

Must be peculiar to the LM-2. I don't have that issue with the SSI-4. It does log a minimum value, however.

Session: Session 14               
time         Lambda    RPM      TPS      Temp   SSI4_4
(sec)        (Lambda) (RPM)   (Deg)   (Temp)   (Volt)
0             0.958     1293         4.8      95.1       0.01
0.08192   0.956     1293         4.8      95.6       0.01
0.16384   0.954     1284         4.8      96.1       0.01
0.24576   0.947     1258         4.8      95.6       0.01
0.32768   0.94       1232         4.8      95.1       0.01
0.4096     0.935     1232         4.8      94.9       0.01
0.49152   0.939     1232         4.8      94.5       0.01
0.57344   0.951     1249         4.8      94.7       0.01
0.65536   0.955     1267         4.8      94.9       0.01


I tried Bernd's converter, but it cocantenated all the log sessions into one big CSV file. Not useful for me. If I need to produce a CSV or DIF file, I just export them from Logworks.

[swooshdave]

Has anyone taken another approach to try to find the programmer(s) from Guzzi who wrote the code and see if they could help. If still at Guzzi they may not be able to help but if retired or moved somewhere else they might.

[Meinolf]

Hi Dave,

I have not tried nor would I expect an answer. The source code would certainly be considered to be IP of Marelli and not be disclosed. And past inquiries about trivial specs for Marelli injectors were answered neither by Marelli HQ or the motor sport group in the UK.

More so because it seems that Marelli is using a library for the code used in their ECUs, beginning with the 15M/RC, the 59M and the 5AM. Some weeks ago I was asked to look at a 59M XDF, some more details here: https://www.ducati.ms/forums/241-tech-forum/715211-59m-ecu-xdf-st4s-800-1000ss-still-interest.html

In fact I have wondered during dark moments if and when a Marelli lawyer would contact me. I'm not sure if the reverse engineering and disclosure of the findings is proper.

Cheers
Meinolf

[swooshdave]

Reverse engineering for profit might raise some eyebrows but as a hobby, probably not. Especially if it’s for the older bikes.

You’re not creating a competitive advantage for any other manufacturer nor are you doing anything a competitor hasn’t done with a lot more resources, not that they may even bother.

Carry on!

[Tusayan]

In fact I have wondered during dark moments if and when a Marelli lawyer would contact me. I'm not sure if the reverse engineering and disclosure of the findings is proper.

People have been doing it since the late 80s without issue.  A guy employed by Duane Mitchell (FIM) in Australia may have been the first to decode the WM firmware of that time.  Others have followed ever since then, more or less keeping up with  Marelli.

 BTW, Australia has always been a hotbed of aftermarket car and bike ECU activity, not least because the products could be sold into the US and Europe without fear of being chased down and fined by local government - which is a bigger issue, particularly in places like California.

Regardless, do carry on 

[Meinolf]

Hi,

ok, so no immediate danger of getting thrown into jail. Which is good, as I did dabble around a bit with a DUC BIN and a XDF for a iaw59M:
https://www.ducati.ms/forums/241-tech-forum/715211-59m-ecu-xdf-st4s-800-1000ss-still-interest.html

More for the fun of it than anything else, as I don't have a bike using a 59M.

And here are the links to the One and Two Lambda XDFs for Guzzi CARCs

One Lambda: https://drive.google.com/open?id=1r_r4iD9WROsOLqgHpiHfEthMkyeACaVU
Two Lambda: https://drive.google.com/open?id=1a_stCXFRzbcmTZGxlUSaSqStGSj4Dnjq

Cheers
Meinolf

[Meinolf]

Hi,

it's time for an update, I guess. The last couple of months less time than usual was spent on the 5AM exploration and building of the ultimate BIN for 2V CARCs, as dry storage room was urgently required for the motorcycles. The beautiful sunny weather will eventually end. So trees were felled, roots whittled down and plaster laid.




















The foundations for the carport are ready, building the carport is the next step.

But nevertheless, for recreational purposes the 5AM exploration and constant improvement of the BIN continued...

Cheers
Meinolf

[Meinolf]

Hi,

the database program, programmed by Beard, for analyzing the huge amounts of datapoints collected by the ZT-2, really proved to be a major improvement. The latest enhancement was adding the ability to use voltage input from the TPS via one of the two analog channels instead of continuing to use the standard %-measurement of the TPS, which suppresses individual breakpoints in the low load area.

The result was a BIN in which the AFR values of the two cylinders not only were identical (or close to identical), but also were in line with the target AFR values choosen over the TPS/rpm breakpoints. In German I'd probably use the wording - it runs as smoothly as a sewing machine.

But, a number of circumstances still make visual analysis necessary. The closely spaced breakpoints and the bilinear interpolation used by the code for in-between datapoints make the identification of individual occurences difficult, the are simply hidden in the mass of averaged datapoints.




So, an occasional reset is needed. While it is normal that the fuel maps look like a hilly landscape, neighboring datapoints which differ to much are simply not plausible. Manual identification and changing them to fit into the respective contextual data is occasionally needed.








The resulting BIN was logged and performed well, the normal optimization process can continue based on a more stable basis.

Alas, one area was still not satisfactory. This is the low load area in the range of 4.6-6� TPS and rpms up to 3000min^-1. The reason was known, the idle fuel map replaces the main fuel map in that area, if and when the ECU considers the TPS to be fully closed. However, that is a range, not a single switching over point.

The idle fuel table and its function was already investigated at length on the ECU simulator, but it's function never understood. The x-axis uses a table with rpm values, the y-axis uses steps in the range of 55-200. A recent observation by Gelos, a Guzzi expert from Poland, led to a better understanding. The y-axis uses the steps employed by the stepper motor. In retrospect this seems very obvious. But that's usually the case with hindsight.

So the idle function uses the idle fuel table to move the stepper motor to steer the idle rpm. But under which circumstances and when?

The above led to the decision to re-start a detailed code analysis with IDAPro, very much based on the astonishing code analysis John Th. from the UK has started...

Cheers
Meinolf

[Meinolf]

Hi,

for some time the assumption was that the entire fueling calculation was done by a subroutine at 0x2C8B4. But, my coding skills being non-existent, understanding escaped.














Luckily John brought light into the dark. And now the function is much better understood.

The first step is to decide if main or idle fuel is used. Either one is then used during the rest of the calculation, the delta fuel values a used in both cases.

Then, depending on whether closed loop is active or not, either the learned Long Term and Short Term fuel corrections are applied, or this is skipped and the air temp/pressure and engine temp corrections are calculated.

Then some, not yet fully understood, scaling operations.

Next the warm-up table corrections are used, if the conditions for the warm-up function are met.

Yet another scaling calculation, if closed loop is active, or application of the CO trim (up to a not yet found rpm) if closed loop is disabled. Finally a check versus the max. value for injection and a cutoff if needed.

The result is the fuel injection value for left and right cylinder.

Yet to be researched is how clutch, gear and maybe even brake are included in above...

Cheers
Meinolf

[smdl]

Fascinating!  Thanks, both for your continued effort on this, and for keeping us updated.

Cheers,
Shaun

[Meinolf]

Hi Shaun,

thanks for your kind words.

This is really a community effort, the main contributors being John and Beard. I'm just the one trying to combine the individual efforts of the ones who know what they are doing and learn something in the process. And the strong urge to share the findings of the countless hours spent on measuring and analyzing to make it worthwhile.

If there's anybody else with some knowledge in reverse engineering assembled code, stand up and be counted :-) Still lot's to learn and explore...

Cheers
Meinolf

[Meinolf]

Hi,

a starting problem on the Norge during a trip to Vienna with temperatures close to 0°C was the reason to look at the use of the parameters used by the code to evaluate battery voltage, even though the starting issue was, it turned out, caused by the battery.

Many, if not all subroutines and parameters used for processing the voltage, were found. One parameter, BatteryVoltageCalc at 0x4C64A, seemed to be the scaling factor applied to the raw ADC values.





So different values were loaded on to the ECU simulator and actual battery voltage, voltage as reported by OBD and on the CANBus and the voltage shown in the dash were noted and evaluated. Quite easy, it turns out, a value of one corresponds to 0.057V. The voltage shown in the dash is not taken from OBD or CANBus, the dashboard uses an independent source. A pity, as the discrepancy between actual voltage and dash voltage can't be set right.

However, as the ignition coils and injectors are acting on voltage related trim tables, and the automatic start on some models also monitors voltage, correcting a discrepancy is an easy blueprinting improvement.

Cheers
Meinolf

[Meinolf]

Hi,

However, as the ignition coils and injectors are acting on voltage related trim tables, and the automatic start on some models also monitors voltage, correcting a discrepancy is an easy blueprinting improvement.

it turned out that the easy bluepring improvement wasn't that easy. A change of the BatteryVoltageCalc factor from 238 to 243 brought the voltage as seen by the ECU in line with the actual voltage, but the bike ran fat - Lambda increased significantly. The assumption was that a voltage trim function for the injectors was the reason.

A few weeks later the assumed trim function was found. The BIN has two trim tables, which govern the pulse length of the injectors depending on the voltage. A Byte value determines which of the two tables is used, that value is fixed in the code, so in fact only one of the two tables is used. And a base value at x04C000 which correlates to the the flow rate of the injector is also present.





The trim table at 0x4C002 (the 2nd one at 0x4C26 isn't used) then determines the pulse width applied to the injector dependent on the voltage.





Cheers
Meinolf

[Omobono]

Hi,

Have you allready find a way where/how to change the idle map.

"Alas, one area was still not satisfactory. This is the low load area in the range of 4.6-6� TPS and rpms up to 3000min-1. The reason was known, the idle fuel map replaces the main fuel map in that area, if and when the ECU considers the TPS to be fully closed. However, that is a range, not a single switching over point."

I have a bike that is running very lean when it is using the idle map, but runs well when it uses the main map. So I would like to tweak the idle map if that is possible.

Ciao
Omo

[Meinolf]

Hi Omo,

Have you allready find a way where/how to change the idle map.

as visible in the flow chart above the idle fuel table is located at 0x4D7B0. X-axis are the stepper motor steps, y-axis the Idle Fuel rpm-values.

I have a bike that is running very lean when it is using the idle map, but runs well when it uses the main map. So I would like to tweak the idle map if that is possible.

Sure, it's possible, but very difficult. I haven't found a way of matching the stepper values to logged AFR data. There's a flag which disables the idle fuel function, but it's used all over the code, so I've refrained from changing it so far.

How do you know that your bike is running lean when using the idle fuel values and well when using the main fuel values?

Cheers
Meinolf

[Omobono]

Hi Meinolf,

I am assuming that the idle fuel table is in use when PADS shows message idle on its screen, and thats when the lambda correction values are also very high.

When I give it some revs the PADS shows off idle and the lambda correction values come back to normal +/- 5%.

Also if I tweak the main fuel table on the idle range it has no effect on the lambda correction.

This is a twin lambda Stelvio I am playing with.

Ciao Omo