Spam: The other white meat

Media Mover can be pretty CPU intensive- transcoding with FFmpeg can be a substantial hit on your server resources. If you couple this with a large amount of batch processing, you are looking at high loads over a long period of time. And of course, this does not even begin to look at what is going on for your users on the site.

Drupal employs the Throttle module to handle high load situations. It looks at traffic patterns on the site to determine if a throttle condition should be employed. In considering how to limit the activity of Media Mover on a server this is an obvious consideration. One of the things that makes Media Mover (for most use cases) different than the needs of modules effected by Throttle is that it is not bound by the traffic occuring on the site- it may well be a factor, but it is not the only consideration. If a Media Mover configuration is using FFmpeg to transcode files, it is almost exclusively CPU bound. Of course there is the Apache process that kicks off cron, some degree of Drupal processing, and perhaps a MySQL hit or two, but largely FFmpeg wants to make use of the server CPU. Media Mover will cycle through sets of files that it is loading from MySQL, but again, in terms of its total impact, mostly we are looking at FFmpeg hitting the CPU.

In this case, if we want to limit Media Mover from running when a server is starting to get overloaded on its CPU we need to check the CPU load. I’ve written a fairly basic utility that does this on Linux / Unix based servers on Media Mover’s cron processing script. It allows other scripts to still access the API without being throttled, but Cron is stopped if load creeps beyond what you set it for.

You can set the threshold where Media Mover should stop, and select which average you want to use as the threshold- the one, five or 15 minute load averages. Once the load drops below the threshold, the Media Mover Cron runs will start up again.