

Try loading each incoming batch into a temporary table outside your transaction. You get most of the performance benefit with batches of 100 rows as you do with batches of 10 000 rows. The performance reason for using transaction batches is to avoid doing a costly COMMIT for every row. Try reducing the size (the row count) of the batches. If each batch handles its own distinct set of rows, this may (or may not) allow a bit more parallelism. Try using SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED before running your batch in each session. Run EXPLAIN on them to ensure that they have the indexes they need, then add the indexes you need. Speed up the DELETE queries used in your batch process. Running multiple batch load processes in parallel is difficult. What I want is these processes to happen without waiting other transactions(insertions or deletions) that are happening at the moment.

That means that these processes will not run in parallel as each one will wait the other. But my understanding is that if I set innodb_lock_wait_timeout to higher value what it will happen is that each transaction will just wait the other transaction to finish. I can see some solutions recommend to set the innodb_lock_wait_timeout to higher value which will propably eliminate the error. show variables like 'innodb_lock_wait_timeout' => 50.SELECT => repeatable-read, repeatable-read, repeatable-read.It is not related to aws batch, as same problem occurs when I try to do it locally.: () (1205, 'Lock wait timeout exceeded try restarting transaction').When multiple batch processes run on the same time I got the following error:.If I run one batch process no problem occurs.Each batch handles its own distinct set of rows.next, deleting multiple rows to the table.first adding multiple rows to the table.

#1205 lock wait timeout exceeded full
If I view the query being run against the database using SHOW FULL PROCESSLIST, that eventually fails with the above message, and then insert it manually from the MySQL console, it works as expected. However, it is being served on a server which hosts our production databases, so I strongly prefer not to do resets if possible. As this is a dev database with no live applications attached I can't envision any problem. On the database reveals no other active queries. However, when I try to insert directly from the MySQL console, it works just fine. Using Python I am consistently getting an (1205, 'Lock wait timeout exceeded try restarting transaction')Įrror whenever I try to insert into a particular table.
