MEGAN sorting all alignments as unclassified

kmanpearl · January 15, 2021, 10:36pm

I am trying to run megan on tabular BLAST files generated using DIAMOND. All of the alignments are being listed as unclassified. I am not sure what I am doing wrong to cause this error. I have included the text from the megan message window below. Any advice would be much appreciated as I am clearly new to using this program.

DIAMOND appears to have ran successfully producing alignments such as:

HWI-ST1329:367:H94AJADXX:1:1101:4596:2412 XP_013318660.1 56.2 48 21 0 146 3 217 264 9.4e-09 62.8

I am using MEGAN Community Edition (version 6.20.19, built 5 Jan 2021)

Opening startup files
Loading ncbi.map: 2,259,889
Loading ncbi.tre: 2,259,893
Executing: show window=ImportBlast;
Loading ec.map: 8,081
Loading ec.tre: 8,085
Loading eggnog.map: 30,875
Loading eggnog.tre: 30,986
Loading gtdb.map: 240,103
Loading gtdb.tre: 240,107
Loading interpro2go.map: 12,738
Loading interpro2go.tre: 28,689
Loading seed.map: 978
Loading seed.tre: 979
Executing: use cViewer=EC state=false;
Executing: use cViewer=EGGNOG state=false;
Executing: use cViewer=GTDB state=false;
Executing: use cViewer=INTERPRO2GO state=false;
Executing: use cViewer=KEGG state=false;
Executing: use cViewer=SEED state=false;
Executing: load mapFile=’/Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/megan-nucl-map-Jul2020.db’ mapType=MeganMapDB cName=Taxonomy;
Executing: use cViewer=Taxonomy state=true;
Executing: update;
Executing: import blastFile=’/Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/12K1B.fasta.m8.txt’ meganFile=’/Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/12K1B.fasta.m8.rma6’ useCompression=true format=BlastTab mode=BlastX maxMatches=100 minScore=50.0 maxExpected=0.01 minPercentIdentity=0.0 topPercent=10.0 minSupportPercent=0.05 lcaAlgorithm=naive minPercentReadToCover=0.0 minPercentReferenceToCover=0.0 minComplexity=0.0 useIdentityFilter=false readAssignmentMode=readCount fNames=;
Classifications: Taxonomy
Annotating RMA6 file using FAST mode (accession database and first accession per line)
Parsing file: /Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/12K1B.fasta.m8.txt
Total reads: 2,392,416
Alignments: 37,192,530
Initializing binning…
Using ‘Naive LCA’ algorithm for binning: Taxonomy
Binning reads…
Total reads: 2,392,416
With hits: 2,392,416
Alignments: 37,192,530
Assig. Taxonomy: 0
MinSupport set to: 1196
Numb. Tax. classes: 1
Class. Taxonomy: 1
Info: Command completed (1613s): import blastFile=’/Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/12K1B.fasta.m8.txt’ meganFile=’/Users/ke…
Executing: show window=message;
Executing: show window=message;

Daniel · January 17, 2021, 2:52pm

You need to use a protein mapping file, because Diamond aligns to protein reference sequences, a nucleotide mapping file, that is, not megan-nucl-map-Jul2020.db.

Please use megan-map-Jul2020-2.db.zip.

kmanpearl · January 17, 2021, 7:44pm

Hi Daniel, thanks for getting back to me. I have been using the megan map you specified previously and having problems with that so I thought maybe I was using the wrong file. Perhaps you can help me troubleshoot the original problem I was having.

When I use the megan-map-Jul2020-2.db.zip file the program generates an incorrect tree. This is a tree that I previously generated running megan on a small subsection of test data. Ever since then, no matter what BLAST file I use it generates the exact same tree matching the test data and not the specified input file. I have tried re-downloading the files from my remote server as well as un-installing and re-installing MEGAN many times.

I also delete all the rma files generated before uninstalling; however, upon reinstallation these files still appear in the open recent tab of MEGAN. I am not sure if this is somehow related to the issue.

Here is the message output from running it with that file:

Opening startup files
Loading ncbi.map: 2,259,889
Loading ncbi.tre: 2,259,893
Executing: show window=ImportBlast;
Executing: load mapFile=’/Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/megan-map-Jul2020-2.db’ mapType=MeganMapDB cName=EC;
Loading ec.map: 8,081
Loading ec.tre: 8,085
Executing: use cViewer=EC state=true;
Executing: load mapFile=’/Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/megan-map-Jul2020-2.db’ mapType=MeganMapDB cName=EGGNOG;
Loading eggnog.map: 30,875
Loading eggnog.tre: 30,986
Executing: use cViewer=EGGNOG state=true;
Executing: load mapFile=’/Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/megan-map-Jul2020-2.db’ mapType=MeganMapDB cName=GTDB;
Loading gtdb.map: 240,103
Loading gtdb.tre: 240,107
Executing: use cViewer=GTDB state=true;
Executing: load mapFile=’/Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/megan-map-Jul2020-2.db’ mapType=MeganMapDB cName=INTERPRO2GO;
Loading interpro2go.map: 12,738
Loading interpro2go.tre: 28,689
Executing: use cViewer=INTERPRO2GO state=true;
Executing: use cViewer=KEGG state=false;
Executing: load mapFile=’/Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/megan-map-Jul2020-2.db’ mapType=MeganMapDB cName=SEED;
Loading seed.map: 978
Loading seed.tre: 979
Executing: use cViewer=SEED state=true;
Executing: load mapFile=’/Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/megan-map-Jul2020-2.db’ mapType=MeganMapDB cName=Taxonomy;
Executing: use cViewer=Taxonomy state=true;
Executing: update;
Executing: import blastFile=’/Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/12K1B.fasta.m8.txt’ meganFile=’/Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/12K1B.fasta.m8-1.rma6’ useCompression=true format=BlastTab mode=BlastX maxMatches=100 minScore=50.0 maxExpected=0.01 minPercentIdentity=0.0 topPercent=10.0 minSupportPercent=0.05 lcaAlgorithm=naive minPercentReadToCover=0.0 minPercentReferenceToCover=0.0 minComplexity=0.0 useIdentityFilter=false readAssignmentMode=readCount fNames=EC EGGNOG GTDB INTERPRO2GO SEED;
Classifications: Taxonomy, SEED, EGGNOG, GTDB, EC, INTERPRO2GO
Annotating RMA6 file using FAST mode (accession database and first accession per line)
Parsing file: /Users/keenanmanpearl/Desktop/reseach/Mack_Lab/Fox/DIAMOND_MEGAN/12K1B.fasta.m8.txt
Total reads: 2,392,416
Alignments: 37,192,530
Initializing binning…
Using ‘Naive LCA’ algorithm for binning: Taxonomy
Using Best-Hit algorithm for binning: SEED
Using Best-Hit algorithm for binning: EGGNOG
Using ‘Naive LCA’ algorithm for binning: GTDB
Using Best-Hit algorithm for binning: EC
Using Best-Hit algorithm for binning: INTERPRO2GO
Binning reads…
Total reads: 2,392,416
With hits: 2,392,416
Alignments: 37,192,530
Assig. Taxonomy: 2,383,328
Assig. SEED: 1,555
Assig. EGGNOG: 1,268,394
Assig. GTDB: 546
Assig. EC: 1,278,790
Assig. INTERPRO2GO: 1,506,776
MinSupport set to: 1196
Min-supp. changes: 603
Min-supp. changes: 1
Numb. Tax. classes: 236
Numb. SEED classes: 11
Numb. EGG. classes: 1,710
Numb. GTDB classes: 1
Numb. EC classes: 1,096
Numb. INT. classes: 4,346
Class. Taxonomy: 236
Class. SEED: 11
Class. EGGNOG: 1,710
Class. GTDB: 1
Class. EC: 1,096
Class. INTERPRO2GO: 4,346

And this is the tree that it is generating:

Daniel · January 17, 2021, 8:09pm

please give me access to the file and I will take a closer look

kmanpearl · January 18, 2021, 5:27am

Thank you for your help. I have attached a link to the file below. I have had other people successfully analyze this data so I think the problem may be localized to my computer.

Daniel · January 18, 2021, 7:16am

Sorry, I am not sure I understand what the issue is, this looks ok to me. I processed the first 10,000 lines of your file and get a similar taxonomy to what you get. As a sanity check, I opened one of the nodes in the Inspector, grabbed the accession from the alignment and then looked it up on NCBI. NCBI confirms that the accession does indeed map to the taxon to which the read is assigned…

So, the assignment of reads to nodes appears to be working as intended…

Here is the data with all nodes uncollapsed… This looks like a typical first-shot with DIAMOND+MEGAN (subsequent interactive modification of parameters will be required, as usual, to get the best results for specific data):

kmanpearl · January 18, 2021, 7:33am

The test file was generated using the first 10,000 lines of this file which is why your tree is the same. The issue is every single time I run MEGAN using the larger complete file or completely different BLAST files, I get the exact same tree. I have un-installed and re-installed MEGAN many times and still get the same result. I have had other people successfully run MEGAN on these files. I only have this issue on my computer (2019 Macbook Pro with MacOS Big Sur V 11.1).

Daniel · January 18, 2021, 10:06am

My point is that MEGAN appears to be showing you what is in this particular file.
Do you have a file that does not contain reads that align to the displayed taxa, but for which MEGAN shows the displayed taxa?

BTW, the list of recently opened files is kept in a preferences file that is called Megan.def.
If you want to erase everything, including all preferences that Megan “remembers”, then please delete Megan.def. On linux or windows, you will find it in your home directory as .Megan.def and on MacOS it is Library/Preferences/Megan.def

kmanpearl · January 18, 2021, 8:00pm

Most, if not all files should have these taxa, at least at the phylum level. However, all files including the full size one I shared with you should have many many more phylum. The fungi makes up a very small subset. The VAST majority of reads should be bacterial and archaeal which are not showing up at all.

No matter which full size input file I use, MEGAN always displays the same tree which exactly matches the tree generated from the 10,000 line test file. I have seen the tree generated using the full 12K1B file and it is much more extensive than the one I am getting.

I deleted the Megan.def file, un-installed and re-installed, however I am still getting the same incomplete tree.

Daniel · January 19, 2021, 7:26am

I extracted all protein accessions in the file that you provided and all of them map to fungal organisms, without exception. MEGAN is working as it should… Please double-check that you are aligning against the full NCBI-nr protein reference database and not just against the fungal references. (If your sample only contained fungal species and you aligned against NCBI-nr, then you would still see off target alignments to prokaryotes, but this is not the case here.)