Okay I managed to fix it. Here is the solution for people who might have the same issue:

The reason the fuel depleted so quickly is because the chemtank was damaged. So you have to click "rest" and enable "repairs" for the vehicles somewhere in caravan screen.

It's possible to remove pawns from the vehicle by drag and drop under the "Pawns" tabs of the caravan. So you can drag and drop pawn out of the vehicles and split the caravan. You can leave 1 pawn until it's repaired and then send another caravan with chemfuel once it's fixed.