GitLab

This was realized after prototyping and tests by @flothoni.
As in IMGT/JunctionAnalysis, the detection relies on the positions of Cys104 and Phe118/Trp118.
The detection is here in O(n), taking advantage of the already aligned V and J segments.

The current implementation will not give a precise positions when there are insertions or deletions
between Cys104 and the end of the V segment  (or between the start of J segment and Phe118/Trp118).
This could be improved by backtracking the DP matrix.

This further cleans the code and allows to output D1/D2 boxes as well.

One day, we may change '5del' and '3del' fields, their naming is not so consistent.

Bug detected thanks to a full Valgrind.

We do not want to detect twice the same D gene.
Note that we do not currently forbid alleles of a same gene.

Discussion between @flothoni, @mikael-s, and @magiraud.

When a D has already been detected, we do not want to detect anything inside this D.
Before this commit, spurious D detections could happen in the EXTEND_D_ZONE.

Discussion between @flothoni, @mikael-s, and @magiraud.

We need to store sequence_or_rc in the Segmenter.

Previously, we have at many places things like "int *del_DD_left, int *DD_start, int *best_DD, int *DD_end, int *del_DD_right".
This was not so clean and error-prone. Now all these parameters are stored into a ‘AlignBox’ object.
This will lead to further simplifications of the code, better code maintenance, and allow some extensions.

We would like to call that on other places than between the V and the J.

We factorize some computations (seq_left, seq_right, seg_N).
This is the last commit of the day sponsored by the CERNA.

This was not used since at least one year.

There was some duplicate code, not tested nor documented,
to generate 'code_short'. This code is now removed.

The 'code_short' value was only used in the json output,
it will be now directly computed by the web application.

core/segment.h, core/windowExtractor.cpp, vidjil.cpp: option '-uu', do not create files for segmented sequences

We start the file creation at STATS_FIRST_UNSEG.

Starting from the full read, we can not limit the DP computations
to a k-band around the diagonal without first knowing where is exactly the junction.

Nevertheless, we can avoid computing about one half of the DP matrix,
as the end of the V / the start of the J (minus some deletions) must be matched.

The BOTTOM_TRIANGLE_SHIFT is now set to 20, and this should be large enough
to handle V/J deletions until ~30 bp (see comment in segment.h).
(The current tests were even passing with BOTTOM_TRIANGLE_SHIFT set to 10.)

Now the FineSegmenter (as launched by 'make shouldvdj_with_rc_merged') is about 35% faster.

K and lambda are now roughly estimated.

There were several problems in the previous implementation. Now:
 - We handle separetly numbers of nucleotides in seq_{left,right} and in ref_{left,right};
 - There is no need to handle equality cases (if is the responsability of the segment_cost);
 - There is no need anymore to call dp.backtrack() (we directly use dp.best_score_on_i).
 - We also properly update del_D_left and del_D_right for VDJ recombinations

The tests added in 3c138657 are now passing.

This better reflects what we find in the reads:
UNSEG_TOO_FEW_V actually means that the J segment was detected, above
the e-value threshold.

Waiting for a true p-value computation, we just require here that
the score is at least the one of 10 matches.

This could be used by the FineSegmenter as well.

delta_max is now useless. It was use to make sure we didn't segment
something not robust enough. Now we use the e-value, which is a much
trustable value.

delta_min is useless to the KmerSegmenter (we can't have overlaps
with the first_max, last_max algorithms).
But it can still be useful to FineSegmenter, that's why we keep it.

Since 40c17d5e and eafb88cb (before the 2015.05 release !), UNSEG_NOISY is not used.

In situations where there are very few k-mers, we prefer to report UNSEG_TOO_FEW_ZERO,
being more consistent to what is done with the e-value check.

As we have now the e-value check on the KmerSegmenter, there is no need to have
a more stringent value here. Lowering this value enables to have better reports
of unsegmentation causes.