Resizing images on VPS


I have several Drupal installations and and 800MB memory on VPS and I seem to have a problem with image manipulation. I tried both GD and Imagemagick and neither of them work if I upload images that are larger than 2MB. The server immediately runs out of memory and shuts down. What’s the solution here? Shall I raise the memory even higher? Do I need like 2GB to resize 5-10MB images in Drupal? Or shall I just go for a dedicated server, would that be a solution? Thanks