Construct the dice from a supply of different coloured faces, with different meanings. The aim is to get a die from your side to their home point.
On your turn, move a die up to 1 + num black sides. (Either an existing die, or a new one) If that moves into another die; stop short, and you both roll.
Black is worst, it wins against nothing. Blue is defense, it wins against nothing; but nothing wins against it. Yellow is best, it wins against white and black and red. White is middling; winning against black and red, losing to yellow. Red is special - it wins against black; but it has the special effect of causing whatever the opponent rolled to be replaced. By black unless it was already black, in which case it goes red.
If you lose, your die is removed from play. I think it can come back some number of turns later; rather than being gone forever.
I think you should have slightly more black to place than you have dice; and a minimum of one per die. And not enough of each other colour to make any die be fully that colour (apart form the one black). And yellow should be rarer than white, obviously.