Prepare both boxes of vanilla pudding using only 3 1/2 cups milk. Stir in Cool Whip into pudding mixture. Blend well. Put 1/2 mixture evenly on top of graham crackers. Lay another layer of graham cracker over pudding using 2nd pkg. of graham crackers. Put rest of pudding mixture evenly over graham crackers. Lay last layer of crackers over pudding.
Put icing in microwave until just spreadable about 10-20 seconds. Spread evenly over top layer of graham crackers.