You said the amp was rated at 6V and required 70mA max. I cannot find it on the ALLelectronics site, but a consumption of less than half a watt seems a bit small.
The individual components seemed to work happily on separate supplies, yet the batteries failed after a few playings (a few minutes?). So something must be passing a heck of a lot of current. Could it be that combining the amp and the other circuits by feeding them from a single supply has created an extra load somehow? I think you need to check the current drawn from the 4.5V tapping and from the full 6V of the battery to see what the culprit is.
I don’t like the idea of tapping the third cell for a lower voltage takeoff, because the three cells will discharge faster than the fourth cell. A better method would be to provide a battery for the highest voltage (the amp) and use a buck converter to change that voltage to a lower one for the USB card/motor.
A buck converter is a switch mode device with an adjustable output voltage and is more efficient than a dropping resistor or series regulator. Converting 6V down to 3V with a buck converter is about 92% to 95% efficient, as against 50% of a series dropper. The converters are inexpensive and reliable.
http://www.ebay.co.uk/itm/DC-DC-Buck-Co ... 27e663ec1bIf you want to stay with alkaline cells then it’s best to use C types as Ken suggested. But if you want to use rechargeables, then I would suggest Li-ion 18650 cells rather than NiMH cells. An 18650 gives 3000mAh to 5000mAh at a nominal 3.7V (3.5V – 4v), which is more than 5 times the energy capacity of an AA NiMH which gives only 2000mAh at a nominal 1.35V.
So you could use two 18650s in series to power the amp, with a buck converter taking the voltage down to a suitable level for the USB/motor combo.
As you have had the motor working OK on 2AA, then if the USB will run on 3.5V, an alternative would be to use a single cell 18650 (or 2 in parallel for extra capacity) for the USB/motor and use a boost converter to raise its voltage high enough for the amp.
http://www.ebay.co.uk/itm/1Pcs-XL6009-D ... 3f389c4c9aI hope that may be of help.