It doesn't matter whether an opponent has less life than they started the turn with. If a player lost 4 life but also gained 6 life, that player still lost 4 life.
Savage Gorger looks at your entire turn to determine whether an opponent lost any life. It doesn't matter how many opponents lost life or how much life they lost, only whether any of them did. It also doesn't matter whether Savage Gorger was on the battlefield when they lost life.