Sam2RMA fails with String or BLOB exceeds size limit

Bug Reports
1

Hello,

I’ve written a wrapper script that runs Megan 6 on my long read sam files. All other files complete fine, bit this one fails with the error pasted below.

The same was obtained by mapping the FASTQ to a diamond database of NCBI nr roughly following this workflow for Nanopore long reads…

Any debugging tips appreciated. Thanks!

Current SAM file: bams/aligned/NCBI_nr/barcode03.sam
Reads file:   fastqs/decontaminated/barcode03.fastq
Output file:  megan_diamond/rmas/barcode03.rma
Classifications: Taxonomy, SEED, EGGNOG, GTDB, EC, INTERPRO2GO
Generating RMA6 file Parsing matches
Annotating RMA6 file using FAST mode (accession database and first accession per line)
Parsing file barcode03.sam
Parsing file: bams/aligned/NCBI_nr/barcode03.sam
Input domination filter: MinPercentCoverToStronglyDominate=90.0 and TopPercentScoreToStronglyDominate=90.0
10% 20% 30% Caught:
org.sqlite.SQLiteException: [SQLITE_TOOBIG] String or BLOB exceeds size limit (statement too long)
        at org.xerial.sqlitejdbc@3.42.0.0/org.sqlite.core.DB.newSQLException(DB.java:1179)
        at org.xerial.sqlitejdbc@3.42.0.0/org.sqlite.core.DB.newSQLException(DB.java:1190)
        at org.xerial.sqlitejdbc@3.42.0.0/org.sqlite.core.DB.throwex(DB.java:1150)
        at org.xerial.sqlitejdbc@3.42.0.0/org.sqlite.core.NativeDB.prepare_utf8(Native Method)
        at org.xerial.sqlitejdbc@3.42.0.0/org.sqlite.core.NativeDB.prepare(NativeDB.java:126)
        at org.xerial.sqlitejdbc@3.42.0.0/org.sqlite.core.DB.prepare(DB.java:264)
        at org.xerial.sqlitejdbc@3.42.0.0/org.sqlite.jdbc3.JDBC3Statement.lambda$executeQuery$1(JDBC3Statement.java:81)
        at org.xerial.sqlitejdbc@3.42.0.0/org.sqlite.jdbc3.JDBC3Statement.withConnectionTimeout(JDBC3Statement.java:454)
        at org.xerial.sqlitejdbc@3.42.0.0/org.sqlite.jdbc3.JDBC3Statement.executeQuery(JDBC3Statement.java:79)
        at megan/megan.accessiondb.AccessAccessionMappingDatabase.getValues(AccessAccessionMappingDatabase.java:220)
        at megan/megan.rma6.RMA6FromBlastCreator.parseFiles(RMA6FromBlastCreator.java:257)
        at megan/megan.tools.SAM2RMA6.createRMA6FileFromSAM(SAM2RMA6.java:340)
        at megan/megan.tools.SAM2RMA6.run(SAM2RMA6.java:307)
        at megan/megan.tools.SAM2RMA6.main(SAM2RMA6.java:69)
2

Hi @bioinfodonk,

Would it be possible to share this file? I also recommend using MEGAN7. You can upload the file to a drive and share the link with us.

Best regards,
Anupam

3

Hi @Anupam,

The files are very large (~20Gb for the FASTQ and SAM each), but I’m working on it. Unfortunately using MEGAN7 isn’t currently possible in my pipeline.

Appreciate any tips you might have.
Thanks

4

Hello @Anupam,

Here’s the link: Dropbox

5

Thanks @bioinfodonk, will update you soon.

Best
Anupam