Skip to content
GitLab
Commit 1e7f27f4 authored by GUILLEVIC Aurore's avatar GUILLEVIC Aurore
Browse files

proofreading aurore

parent 039f989f
Hide whitespace changes
Inline Side-by-side
rsa250/README.md
View file @ 1e7f27f4
......@@ -10,20 +10,20 @@ This is exactly similar to RSA-240.
The polynomial selection size-optimization
step was performed with cado-nfs in client/server mode, using a myslq database:
```
```shell
./cado-nfs.py params.rsa250 database=db:mysql://USERNAME:PASSWORD@localhost:3306/rsa250
```
with the file [`params.rsa250`](params.rsa250).
The root-optimization step was performed on the best 100 size-optimized
polynomials (those with the smallest exp_E value) with the following command:
```
polyselect_ropt -inputpolys candidates -Bf 2.1e9 -Bg 1.8e9 -area 2.4e19 -ropteffort 10 -t 100
```shell
$CADO_BUILD/polyselect/polyselect_ropt -inputpolys candidates -Bf 2.1e9 -Bg 1.8e9 -area 2.4e19 -ropteffort 10 -t 100
```
And the winner is:
```
```shell
cat > rsa250.poly <<EOF
n: 2140324650240744961264423072839333563008614715144755017797754920881418023447140136643345519095804679610992851872470914587687396261921557363047454770520805119056493106687691590019759405693457452230589325976697471681738069364894699871578494975937497937
poly0: -3256571715934047438664355774734330386901,185112968818638292881913
......@@ -39,22 +39,22 @@ EOF
For estimating the number of relations produced by a set of parameters:
- We compute the corresponding factor bases. In our case, the side 0 is
rational, so there is no need to precompute the factor base.
- We create a "hint" file that we tell which strategy to use for which
- We create a "hint" file where we write which strategy to use for which
special-q size.
- We random-sample in the global q-range, using sieving and not batch:
this produces the same relations. This is slower but -batch is
currently incompatible with on-line duplicate removal.
- We random-sample in the global q-range, using sieving but without batch,
this produces the same relations. This is slower but `-batch` is
currently incompatible with on-line (on-the-fly) duplicate removal.
Here is what it gives with final parameters used in the computation:
Here is what it gives with the final parameters used in the computation:
```
```shell
$CADO_BUILD/sieve/makefb -poly rsa250.poly -side 1 -lim 2147483647 -maxbits 16 -t 8 -out rsa250.fb1.gz
```
The hint file has a weird format. The following basically says "Three
algebraic large primes for special-q less than 2^31, and two otherwise."
```
```shell
cat > rsa250.hint <<EOF
30@1 1.0 1.0 A=33 2147483647,36,72,2.2 2147483647,37,111,3.2
31@1 1.0 1.0 A=33 2147483647,36,72,2.2 2147483647,37,111,3.2
......@@ -67,7 +67,7 @@ EOF
We can now sieve for random-sampled special-q, removing duplicate
relations on-the-fly.
```
```shell
$CADO_BUILD/sieve/las -poly rsa250.poly -fb1 rsa250.fb1.gz -lim0 2147483647 -lim1 2147483647 -lpb0 36 -lpb1 37 -q0 1e9 -q1 12e9 -dup -dup-qmin 0,1000000000 -sqside 1 -A 33 -mfb0 72 -mfb1 111 -lambda0 2.2 -lambda1 3.2 -random-sample 1024 -t auto -bkmult 1,1l:1.25975,1s:1.5,2s:1.1 -v -bkthresh1 80000000 -adjust-strategy 2 -fbc /tmp/fbc -hint-table rsa250.hint
```
......@@ -78,12 +78,13 @@ special-q in the global q-range. The latter can be precisely estimated
using the logarithmic integral function as an approximation of the number
of degree-1 prime ideals below a bound. In Sagemath, this gives:
```
[sage]
```python
# [sage]
ave_rel_per_sq = 12.7 ## pick value ouput by las
number_of_sq = log_integral(12e9) - log_integral(1e9)
tot_rels = ave_rel_per_sq * number_of_sq
print (tot_rels)
# we obtain 6.23205306433878e9
```
This estimate of 6.2e9 can be made more precise by increasing the number of
special-q that are sampled for sieving. It is also possible to have
......@@ -107,7 +108,7 @@ to start a first run and interrupt it as soon as the cache is written.
In order to measure the cost of sieving in the special-q subrange where
sieving is used on both sides, the typical command-line is as follows:
```
```shell
time $CADO_BUILD/sieve/las -poly rsa250.poly -fb1 rsa250.fb1.gz -lim0 2147483647 -lim1 2147483647 -lpb0 36 -lpb1 37 -q0 1e9 -q1 4e9 -sqside 1 -A 33 -mfb0 72 -mfb1 111 -lambda0 2.2 -lambda1 3.2 -random-sample 1024 -t auto -bkmult 1,1l:1.25975,1s:1.5,2s:1.1 -v -bkthresh1 80000000 -adjust-strategy 2 -fbc /tmp/fbc
```
......@@ -131,12 +132,13 @@ case, since there are 32 cores and we sieved 1024 special-qs, this gives
Finally, it remains to multiply by the number of special-q in this
subrange. We get (in Sagemath):
```
[sage]
```python
# [sage]
cost_in_core_sec=(log_integral(4e9)-log_integral(1e9))*(128*60+8.1)*32/1024
cost_in_core_hours=cost_in_core_sec/3600
cost_in_core_years=cost_in_core_hours/24/365
print (cost_in_core_hours, cost_in_core_years)
# (9.28413860267641e6, 1059.83317382151)
```
With this experiment, we get therefore about 1060 core.years for this sub-range.
......@@ -144,23 +146,23 @@ With this experiment, we get therefore about 1060 core.years for this sub-range.
#### Cost of 1-sided sieving + batch in the q-range [4e9,12e9]
For special-qs larger then 4e9, since we are using batch smoothness
For special-qs larger than 4e9, since we are using batch smoothness
detection on side 0, we have to precompute the rsa250.batch0 file that
contains the product of all primes to be extracted. (Note the -batch1
contains the product of all primes to be extracted. (Note the `-batch1`
option is mandatory, even if for our parameters, no file is produced on
side 1.)
```
```shell
$CADO_BUILD/sieve/ecm/precompbatch -poly rsa250.poly -lim0 0 -lim1 2147483647 -batch0 rsa250.batch0 -batch1 rsa250.batch1 -batchlpb0 31 -batchlpb1 30
```
Then, we can use the [`sieve-batch.sh`](sieve-batch.sh) shell-script given in this
repository. This launches:
- one instance of las, that does the sieving on side 1 and print the
- one instance of `las`, that does the sieving on side 1 and prints the
survivors to files;
- 6 instances of the `finishbatch` program. Those instances process the files as they are
produced, do the batch smoothness detection, and produce relations.
They take as input on the command-line two arguments -q0 xxx -q1 xxx
They take as input on the command-line two arguments `-q0 xxx -q1 xxx`
describing the range of special-q to process.
In order to run it on your own machine, there are some variables to
......@@ -170,40 +172,42 @@ can also be adjusted depending on the number of cores available on the
machine.
When the paths are properly set, here is a typical invocation:
```
```shell
./sieve-batch.sh -q0 4000000000 -q1 4000100000
```
The script prints on stdout the start and end date, and in the output of
las that can be found in $result_dir/log/las.${q0}-${q1}.out , the number
`las` that can be found in `$result_dir/log/las.${q0}-${q1}.out`, the number
of special-q that have been processed can be found. From this one can
again deduce the cost in core.seconds to process one special-q and then
the overall cost of sieving the q-range [4e9,12e9].
The design of this script imposes to have a rather long range of
special-q to handle for each run of sieve-batch.sh . Indeed, during the
last minutes, the finishbatch jobs need to take care of the last survivor
files while las is no longer running, so that the node is not fully
occupied. If the sieve-batch.sh job takes a few hours, this fade-out
special-q to handle for each run of `sieve-batch.sh`. Indeed, during the
last minutes, the `finishbatch` jobs need to take care of the last survivor
files while `las` is no longer running, so that the node is not fully
occupied. If the `sieve-batch.sh` job takes a few hours, this fade-out
phase takes negligible time. Both for the benchmark and in production it
is then necessary to have jobs taking at least a few hours.
On our sample machine, here is an example of a benchmark:
```
```shell
./sieve-batch.sh -q0 4000000000 -q1 4000100000 > /tmp/sieve-batch.out
[ wait ... ]
# [ wait ... ]
start=$(date -d "`grep "^Starting" /tmp/sieve-batch.out | head -1 | awk -F " at " '//{print $2}'`" +%s)
end=$(date -d "`grep "^End" /tmp/sieve-batch.out | tail -1 | awk -F " at " '//{print $2}'`" +%s)
nb_q=`grep "# Discarded 0 special-q's out of" /tmp/log/las.4000000000-4000100000.out | awk '{print $(NF-1)}'`
echo -n "Cost in core.sec per special-q: "; echo "($end-$start)/$nb_q*32" | bc -l
Cost in core.sec per special-q: 116.51841746248294679392
[sage]
```
``` python
# [sage]
cost_in_core_sec=(log_integral(12e9)-log_integral(4e9))*116.5
cost_in_core_hours=cost_in_core_sec/3600
cost_in_core_years=cost_in_core_hours/24/365
print (cost_in_core_hours, cost_in_core_years)
# (1.13780852428233e7, 1298.86817840449)
```
With this experiment, we get 116.5 core.sec per special-q, and therefore
......@@ -211,8 +215,10 @@ we obtain about 1300 core.years for this sub-range.
## Simulating the filtering output
Just use the script `scripts/estimate_matsize.sh` available in the
cado-nfs repository.
Just use the script
[`scripts/estimate_matsize.sh`](../../cado-nfs/scripts/estimate_matsize.sh)
# https://gitlab.inria.fr/cado-nfs/cado-nfs/-/blob/master/scripts/estimate_matsize.sh
available in the cado-nfs repository.
This feature is still experimental and we do not claim full
reproducibility for this part of our work which was useful only to help
......@@ -224,7 +230,7 @@ us choosing the parameters, but not for the proper computation.
The benchmark command-lines above can be used almost as-is for
reproducing the full computation. It is just necessary to remove the
-random-sample option and to adjust the -q0 and -q1 to create many small
`-random-sample` option and to adjust the `-q0` and `-q1` to create many small
work units that in the end cover exactly the global q-range.
Since we do not expect anyone to spend again as much computing resources
......@@ -248,21 +254,22 @@ for the estimates in previous sections, and extrapolate.
Several filtering experiments were done during the sieving phase.
The final one can be reproduced as follows, with revision f59dcf48f:
```
purge -out purged3.gz -nrels 6132671469 -keep 160 -col-min-index 0 -col-max-index 8460769956 -t 56 -required_excess 0.0 files
```shell
$CADO_BUILD/filter/purge -out purged3.gz -nrels 6132671469 -keep 160 -col-min-index 0 -col-max-index 8460769956 -t 56 -required_excess 0.0 files
```
where `files` is the list of files with unique relations (output of `dup2`).
This took about 9.3 hours on the machine wurst, with 138GB of peak memory.
This took about 9.3 hours on the machine wurst (Xeon E7-4850 v3 @ 2.20GHz, 56
physical cores, 112 virtual cores, 4 NUMA nodes, 1.5TB memory), with 138GB of peak memory.
The merge step can be reproduced as follows (revision eaeb2053d):
```
merge -out history5 -t 112 -target_density 250 -mat purged3.gz -skip 32
```shell
$CADO_BUILD/filter/merge -out history5 -t 112 -target_density 250 -mat purged3.gz -skip 32
```
and took about 3 hours on the machine wurst, with a peak memory of 1500GB.
Finally the replay step can be reproduced as follows:
```
replay -purged purged3.gz -his history5 -out rsa250.matrix.250.bin
```shell
$CADO_BUILD/filter/replay -purged purged3.gz -his history5 -out rsa250.matrix.250.bin
```
## Estimating linear algebra time more precisely, and choosing parameters
......@@ -271,11 +278,11 @@ replay -purged purged3.gz -his history5 -out rsa250.matrix.250.bin
## Reproducing the characters step
Let `W` be the kernel vector computed by the linear algebra step.
Let **W** be the kernel vector computed by the linear algebra step.
The characters step transforms this kernel vector into dependencies.
We used the following command on the machine `wurst`:
```
characters -poly rsa250.poly -purged purged3.gz -index rsa250.index.gz -heavyblock rsa250.matrix.250.dense.bin -out rsa250.kernel -ker K.sols0-64.0 -lpb0 36 -lpb1 37 -nchar 50 -t 56
```shell
$CADO_BUILD/linalg/characters -poly rsa250.poly -purged purged3.gz -index rsa250.index.gz -heavyblock rsa250.matrix.250.dense.bin -out rsa250.kernel -ker K.sols0-64.0 -lpb0 36 -lpb1 37 -nchar 50 -t 56
```
This gave after about 2 hours 23 dependencies.
......@@ -283,17 +290,17 @@ This gave after about 2 hours 23 dependencies.
The following command line can be used to produce the actual dependencies
(rsa250.dep.000.gz to rsa250.dep.022.gz) from the `rsa250.kernel` file:
```
sqrt -poly rsa250.poly -prefix rsa250.dep.gz -purged purged3.gz -index rsa250.index.gz -ker rsa250.kernel -t 56 -ab```
```shell
$CADO_BUILD/sqrt/sqrt -poly rsa250.poly -prefix rsa250.dep.gz -purged purged3.gz -index rsa250.index.gz -ker rsa250.kernel -t 56 -ab
```
Then the following command line can be used to compute the square root on the
rational side of dependency `nnn`:
```
sqrt -poly rsa250.poly -prefix rsa250.dep -dep nnn -side0
```shell
$CADO_BUILD/sqrt/sqrt -poly rsa250.poly -prefix rsa250.dep -dep nnn -side0
```
and similarly on the algebraic side:
```
sqrt -poly rsa250.poly -prefix rsa250.dep -dep nnn -side1
```shell
$CADO_BUILD/sqrt/sqrt -poly rsa250.poly -prefix rsa250.dep -dep nnn -side1
```
Then we can simply compute `gcd(x-y,N)` where `x` and `y` are the square
roots on the rational and algebraic sides respectively, and `N` is RSA-250.
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment