In short, I think Gakato is right.

In long: I haven't tested this specific case, but I'm fairly certain the game treats Retaliate:Fireball as it would treat a melee retaliation (for example, assuming no first strike / slow, when you hit a lower-level monster, you attack it first, and if it survives your attack, it can retaliate against you).
So what I believe the order of events to be is:
1. Your fireball attack deals damage
2. Monster retaliates against your attack (if your attack didn't kill it)
3. Your previous burn stacks pop
...in your example, this is where the kill would happen, and then you'd get the XP for it. So you are probably dead by then.
4. New burn stacks get applied to the monster you just fireballed
...etc.
Now, I do know for sure that leveling up happens after combat. So in this sense the above described order isn't even relevant, because you will gain XP for your kill as soon as it happens, but you will only level up after combat (if you gained enough XP to level up). This is what can be exploited to trigger double-kills, gaining XP based on your current level for two simultaneous kills, even if either of those kills would level you up.
"Thinker", just without the "ache".