Additional languages
MySQL 3.23, 4.0, 4.1 Reference Manual :: 8 Client and Utility Programs :: 8.10 mysqlbinlog — Utility for Processing Binary Log Files
« 8.9 mysqladmin — Client for Administering a MySQL Server
8.11 mysqlcheck — A Table Maintenance and Repair Program »
Section Navigation      [Toggle]

8.10. mysqlbinlog — Utility for Processing Binary Log Files

The binary log files that the server generates are written in binary format. To examine these files in text format, use the mysqlbinlog utility. It is available as of MySQL 3.23.14. You can also use mysqlbinlog to read relay log files written by a slave server in a replication setup. Relay logs have the same format as binary log files.

Invoke mysqlbinlog like this: 

shell> mysqlbinlog [options] log_file ...

For example, to display the contents of the binary log file named binlog.000003, use this command: 

shell> mysqlbinlog binlog.0000003

The output includes all events contained in binlog.000003. Event information includes the statement executed, the time the statement took, the thread ID of the client that issued it, the timestamp when it was executed, and so forth.

The output from mysqlbinlog can be re-executed (for example, by using it as input to mysql) to reapply the statements in the log. This is useful for recovery operations after a server crash. For other usage examples, see the discussion later in this section.

Normally, you use mysqlbinlog to read binary log files directly and apply them to the local MySQL server. It is also possible to read binary logs from a remote server by using the --read-from-remote-server option. When you read remote binary logs, the connection parameter options can be given to indicate how to connect to the server. These options are --host, --password, --port, --protocol, --socket, and --user; they are ignored except when you also use the --read-from-remote-server option.

Binary logs and relay logs are discussed further in Section 5.11.4, “The Binary Log”, and Section 6.3.4, “Replication Relay and Status Files”.

mysqlbinlog supports the following options:

  • --help, -?

    Display a help message and exit.

  • --character-sets-dir=path

    The directory where character sets are installed. See Section 5.10.1, “The Character Set Used for Data and Sorting”.

  • --database=db_name, -d db_name

    List entries for just this database (local log only). You can only specify one database with this option - if you specify multiple --database options, only the last one is used. This option forces mysqlbinlog to output entries from the binary log where the default database (that is, the one selected by USE) is db_name. Note that this does not replicate cross-database statements such as UPDATE some_db.some_table SET foo='bar' while having selected a different database or no database.

  • --debug[=debug_options], -# [debug_options]

    Write a debugging log. A typical debug_options string is often 'd:t:o,file_name'.

  • --disable-log-bin, -D

    Disable binary logging. This is useful for avoiding an endless loop if you use the --to-last-log option and are sending the output to the same MySQL server. This option also is useful when restoring after a crash to avoid duplication of the statements you have logged. This option is available as of MySQL 4.1.8.

    This option requires that you have the SUPER privilege. It causes mysqlbinlog to include a SET SQL_LOG_BIN=0 statement in its output to disable binary logging of the remaining output. The SET statement is ineffective unless you have the SUPER privilege.

  • --force-read, -f

    With this option, if mysqlbinlog reads a binary log event that it does not recognize, it prints a warning, ignores the event, and continues. Without this option, mysqlbinlog stops if it reads such an event.

  • --host=host_name, -h host_name

    Get the binary log from the MySQL server on the given host.

  • --local-load=path, -l path

    Prepare local temporary files for LOAD DATA INFILE in the specified directory.

  • --offset=N, -o N

    Skip the first N entries in the log.

  • --password[=password], -p[password]

    The password to use when connecting to the server. If you use the short option form (-p), you cannot have a space between the option and the password. If you omit the password value following the --password or -p option on the command line, you are prompted for one.

    Specifying a password on the command line should be considered insecure. See Section 5.8.6, “Keeping Your Password Secure”.

  • --port=port_num, -P port_num

    The TCP/IP port number to use for connecting to a remote server.

  • --position=N, -j N

    Deprecated. Use --start-position instead (starting from MySQL 4.1.4).

  • --protocol={TCP|SOCKET|PIPE|MEMORY}

    The connection protocol to use. Added in MySQL 4.1.

  • --read-from-remote-server, -R

    Read the binary log from a MySQL server rather than reading a local log file. Any connection parameter options are ignored unless this option is given as well. These options are --host, --password, --port, --protocol, --socket, and --user.

  • --result-file=name, -r name

    Direct output to the given file.

  • --set-charset=charset_name

    Add a SET NAMES charset_name statement to the output to specify the character set to be used for processing log files. This option was added in MySQL 4.1.21.

  • --short-form, -s

    Display only the statements contained in the log, without any extra information.

  • --socket=path, -S path

    For connections to localhost, the Unix socket file to use, or, on Windows, the name of the named pipe to use.

  • --start-datetime=datetime

    Start reading the binary log at the first event having a timestamp equal to or later than the datetime argument. The datetime value is relative to the local time zone on the machine where you run mysqlbinlog. The value should be in a format accepted for the DATETIME or TIMESTAMP data types. For example: 

    shell> mysqlbinlog --start-datetime="2005-12-25 11:25:56" binlog.000003

    This option is available as of MySQL 4.1.4. It is useful for point-in-time recovery. See Section 5.9.2, “Example Backup and Recovery Strategy”.

  • --stop-datetime=datetime

    Stop reading the binary log at the first event having a timestamp equal or posterior to the datetime argument. This option is useful for point-in-time recovery. See the description of the --start-datetime option for information about the datetime value. This option is available as of MySQL 4.1.4.

  • --start-position=N

    Start reading the binary log at the first event having a position equal to the N argument. This option applies to the first log file named on the command line. Available as of MySQL 4.1.4 (previously named --position).

  • --stop-position=N

    Stop reading the binary log at the first event having a position equal or greater than the N argument. This option applies to the last log file named on the command line. Available as of MySQL 4.1.4.

  • --to-last-log, -t

    Do not stop at the end of the requested binary log from a MySQL server, but rather continue printing until the end of the last binary log. If you send the output to the same MySQL server, this may lead to an endless loop. This option requires --read-from-remote-server. Available as of MySQL 4.1.2.

  • --user=user_name, -u user_name

    The MySQL username to use when connecting to a remote server.

  • --version, -V

    Display version information and exit.

You can also set the following variable by using --var_name=value syntax:

  • open_files_limit

    Specify the number of open file descriptors to reserve.

It is also possible to set variables by using --set-variable=var_name=value or -O var_name=value syntax. This syntax is deprecated.

You can pipe the output of mysqlbinlog into the mysql client to execute the statements contained in the binary log. This is used to recover from a crash when you have an old backup (see Section 5.9.1, “Database Backups”). For example: 

shell> mysqlbinlog binlog.000001 | mysql

Or: 

shell> mysqlbinlog binlog.[0-9]* | mysql

You can also redirect the output of mysqlbinlog to a text file instead, if you need to modify the statement log first (for example, to remove statements that you do not want to execute for some reason). After editing the file, execute the statements that it contains by using it as input to the mysql program.

mysqlbinlog has the --start-position option, which prints only those statements with an offset in the binary log greater than or equal to a given position (the given position must match the start of one event). It also has options to stop and start when it sees an event with a given date and time. This enables you to perform point-in-time recovery using the --stop-datetime option (to be able to say, for example, “roll forward my databases to how they were today at 10:30 a.m.”).

If you have more than one binary log to execute on the MySQL server, the safe method is to process them all using a single connection to the server. Here is an example that demonstrates what may be unsafe: 

shell> mysqlbinlog binlog.000001 | mysql # DANGER!!
shell> mysqlbinlog binlog.000002 | mysql # DANGER!!

Processing binary logs this way using different connections to the server causes problems if the first log file contains a CREATE TEMPORARY TABLE statement and the second log contains a statement that uses the temporary table. When the first mysql process terminates, the server drops the temporary table. When the second mysql process attempts to use the table, the server reports “unknown table.”

To avoid problems like this, use a single connection to execute the contents of all binary logs that you want to process. Here is one way to do so: 

shell> mysqlbinlog binlog.000001 binlog.000002 | mysql

Another approach is to write all the logs to a single file and then process the file: 

shell> mysqlbinlog binlog.000001 >  /tmp/statements.sql
shell> mysqlbinlog binlog.000002 >> /tmp/statements.sql
shell> mysql -e "source /tmp/statements.sql"

In MySQL 3.23, the binary log did not contain the data to load for LOAD DATA INFILE statements. To execute such a statement from a binary log file, the original data file was needed. Starting from MySQL 4.0.14, the binary log does contain the data, so mysqlbinlog can produce output that reproduces the LOAD DATA INFILE operation without the original data file. mysqlbinlog copies the data to a temporary file and writes a LOAD DATA LOCAL INFILE statement that refers to the file. The default location of the directory where these files are written is system-specific. To specify a directory explicitly, use the --local-load option.

Because mysqlbinlog converts LOAD DATA INFILE statements to LOAD DATA LOCAL INFILE statements (that is, it adds LOCAL), both the client and the server that you use to process the statements must be configured to allow LOCAL capability. See Section 5.6.4, “Security Issues with LOAD DATA LOCAL”.

MySQL Enterprise.  For expert advice on the security implications of enabling LOCAL, subscribe to the MySQL Network Monitoring and Advisory. For more information see http://www.mysql.com/products/enterprise/advisors.html.

Warning: The temporary files created for LOAD DATA LOCAL statements are not automatically deleted because they are needed until you actually execute those statements. You should delete the temporary files yourself after you no longer need the statement log. The files can be found in the temporary file directory and have names like original_file_name-#-#.

Before MySQL 4.1, mysqlbinlog could not prepare output suitable for mysql if the binary log contained interlaced statements originating from different clients that used temporary tables of the same name. This is fixed in MySQL 4.1. However, the problem still existed for LOAD DATA INFILE statements until it was fixed in MySQL 4.1.8.


User Comments

Posted by Tom Mulkins on September 23 2003 12:31pm[Delete] [Edit]

I had some problems using mysqlbinlog with temporary files. It would have helped to have an explanation above but here is a brief example:

mysqlbinlog -d mydb -r mydb.sql mydb-bin.001

/*The above command will create a file called mydb.sql in my CWD(current working directory) with queries extracted from binary log mydb-bin.001 for mydb database queries only*/

Now say I had some load data infile statements in my binary log. If my /tmp directory did not contain those files mysqbinlog would create them for me. Here's th problem, if the file aready exists mysqlbinlog will error out with message File: 'tmp/XXX.csv' not found. Yet if you look in your /tmp directory there it is! Don't panic...mysqlbinlog won't write over an existing file and there is no flag to do so (in my opinion there should be that option).

Now you could delete the files from your /tmp directory and et mysqlbinlog recreate them for you but it is simpler to create a tmp directory in your CWD like this:

mkdir tmp

Now use the mysqlbinlog flag --local-load to specify your CWD/tmp directory to WRITE the files like this:

mysqlbinlog -d mydb -r mydb.sql --local-load="tmp/" mydb-bin.001

Your files will be created in CWD/tmp. Should you need to run the mysqlbinlog utilty again just rm CWD/tmp/* and run the utility again.

Hope this helps,
Tom

Posted by KEvin on February 9 2005 2:06pm[Delete] [Edit]

Some things to know about mysqlbinlog which did not strike me as obvious (also it is hinted by the doc) :

--read-from-remote-server :
1) with this option you can only read files present in binary_log-bin.index on the master so you cannot read relay log files on the distant server
2) the distant mysql server must be up (you cannot just read the distant files), so it loses much of its utility : if the distant master is up you can "start slave" or "change master to MASTER_LOG_FILE=...".
But if the master is down and you want to get the latest changes you must copy the remote (with scp for example) binary logs and then run mysqlbinlog locally ...

--start-position (or --position) :
1) it must be the exact position of an event.
2) it is the first position that will be read so you must not use the "Read_Master_Log_Pos" (as shown by "show slave status") which is the position of the last event done.
You have to use :
--start-position=Read_Master_Log_Pos --offset=1 Master_Log_File
to skip the first event.
As Read_Master_Log_Pos is one of the most easy position to get it is a pity that you have to specify the offset each time...







Posted by john danilson on September 8 2006 6:40pm[Delete] [Edit]

I found the --start-datetime and --stop-datetime to be finicky about the format. While yyyy-mm-dd hh:mm:ss work fine elsewhere, this expected yy-mm-dd hh:mm:ss to work.

Posted by gejo paul on March 27 2007 1:01pm[Delete] [Edit]

mysqlbinlog -R --database=dbname --disable-log-bin --host=MasterIP --port=3306 --socket=/var/lib/mysql/mysql.sock --protocol=TCP --user=root - p --start-position=569016451(current log position in slave) -t


Add your own comment.