BigDump: Staggered MySQL Importer

BigDumpAll I can really say about this awesome little PHP tool is just how much I wish I had a known you sooner.  I recall the last time I moved WordPress sites around in mass from one server to another.  Fortunately at that time I was only really working with two sites that had really large databases and in the end at that time I just did some cutting up myself in smaller sections of SQL stuff in order to get them to run in both a time and memory space that the server would allow.  As I am preparing to that again with some of my stuff and even more importantly on a large-scale with some sites at work I needed to find something else. Enter stage left, BigDump.

Despite the odd and somewhat easy to poke fun at name, it truly is an apt name for this little application.  It is a basic, rudimentary too that gets the job done with little no frills either, which is just exactly how it should be.  With out getting into to many details, at the simplest this tool takes the various SQL statements that rebuilds your exported information when you do an import and breaks it into several smaller pieces.  By doing the smaller pieces, that allows the process to complete without running into the typical limitation that occurs with either an out of memory issue or more likely a timeout due to the length of time things are taking.

You have to set a few options and be away of a couple of things to get the tool working, but it is really simple to do and the file you set that all in is the PHP file with the script itself.  The basic minimum is a set of connection of information to a database – usually name of the database, username, password, and most likely the setting of localhost.  Additional items could include encoding and such, which will be important if you have any of that going.  You can then run the script from your browser and be prompted for selection of any SQL files for input or you could also hard code what files you are going to import as well.

One thing that is important to make it work, when you do your export make sure it is breaking the lines up into separate statements.  You can in theory (and it usually the default for exports) create a table and populate it with all the data in one big long SQL statement.  BigDump does not really have any place to break the file up then though and will usually fail.  It does do a preliminary scan, but you can be ahead of the game if you split it up.

One last tip, the directory on your web server where BigDump is going to reside, needs to be writable to either the group or the owner for whatever user apache (assumed apache web server) is going to be running as.  You can change it later, though you will probably want some subset of that for a WordPress install anyway.  After you do this import, make sure you delete the bigdump.php file and probably not a bad idea to kill off the data file for the import as well.

** – Image from the BigDump website.