readme.renumf90
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
readme.renumf90 [2012/05/28 01:02] – ignacio | readme.renumf90 [2023/03/03 03:59] – [Fields in the parameter file] dani | ||
---|---|---|---|
Line 1: | Line 1: | ||
- | < | + | |
- | ======================================================= | + | =======RENUMF90====== |
- | RENUMF90 - | + | A renumbering program for the '' |
- | | + | Ignacy Misztal and Ignacio Aguilar, University of Georgia\\ |
- | ======================================================= | + | |
- | Ignacy Misztal and Ignacio Aguilar, University of Georgia | + | |
August 27, 2001 - Mar 17, 2011 | August 27, 2001 - Mar 17, 2011 | ||
- | Summary | + | =====Summary===== |
- | ======= | + | |
RENUMF90 is a renumbering program for the BLUPF90 family of programs. It | RENUMF90 is a renumbering program for the BLUPF90 family of programs. It | ||
Line 21: | Line 18: | ||
Warnings | Warnings | ||
- | - input files cannot contain character #. | + | * input files cannot contain character #. |
- | - missing animals have code 0; 00 may be treated as a known animal | + | |
- | + | \\ | |
- | Structure of parameter file | + | **Hint**: type '' |
- | =========================== | + | =====Structure of parameter file====== |
The parameter file contains keywords in capital followed by specifications | The parameter file contains keywords in capital followed by specifications | ||
- | of a given effect/data item. The keywords need to be typed exactly. Specific | + | of a given effect/data item.\\ The keywords need to be typed exactly.\\ Specific |
keywords need to occur sequentially, | keywords need to occur sequentially, | ||
- | Bugs | + | |
- | ==== | + | =====Bugs===== |
IDs starting with " | IDs starting with " | ||
- | Fields in the parameter file | ||
- | ============================= | ||
- | # Parameter file for program renf90; it is translated | + | =====Fields in the parameter file===== |
- | # file for BLUPF90 family of programs. | + | |
+ | # Parameter file for renumf90. It is translated | ||
- | Lines with # are treated as comments | + | Lines with # are treated as comments |
| | ||
- | + | < | |
DATAFILE | DATAFILE | ||
f1 | f1 | ||
+ | </ | ||
+ | The data file is f1 | ||
- | The data file is f1 | + | <file> |
+ | SKIP_HEADER | ||
+ | n | ||
+ | </ | ||
+ | This is optional. It skips the first n lines as header in the data file. | ||
+ | |||
+ | < | ||
TRAITS | TRAITS | ||
t1 t2 .. tn | t1 t2 .. tn | ||
+ | </ | ||
+ | t1-tn are positions of traits in datafile; n defines the number of traits | ||
- | t1-tn are positions of traits in datafile; n defines the number of traits | + | < |
- | + | ||
FIELDS_PASSED TO OUTPUT | FIELDS_PASSED TO OUTPUT | ||
p1 p2 .. pm | p1 p2 .. pm | ||
- | + | </ | |
- | | + | fields p1-pn are passed to output without changes; can be empty |
+ | < | ||
WEIGHT(S) | WEIGHT(S) | ||
- | w | + | w1 [w2 w3...wn] |
- | w is | + | </ |
- | | + | w1 [w2 w3...wn] are |
+ | |||
+ | < | ||
RESIDUAL_VARIANCE | RESIDUAL_VARIANCE | ||
r | r | ||
- | r is matrix of residual (co)variances | + | </ |
+ | r is matrix of residual (co)variances | ||
+ | < | ||
EFFECT | EFFECT | ||
e1.. en type form | e1.. en type form | ||
+ | </ | ||
+ | this line defines one group of effects; e1 .. en are positions of this effect for all traits; | ||
- | this line defines one group of effects; e1 .. en are positions of this | + | positions can be different for each trait for fixed effects; |
- | | + | |
- | effects; | + | |
- | | + | |
- | + | ||
- | type is ' | + | |
- | + | ||
- | form is ' | + | |
+ | for random effects, only one position + 0 (missing) efefct are | ||
+ | possible. | ||
+ | * type is ' | ||
+ | * for crossclassified effects: form is ' | ||
+ | * for covariables: | ||
+ | |||
+ | < | ||
NESTED | NESTED | ||
d1 .. dn form | d1 .. dn form | ||
+ | </ | ||
+ | optional for covariables only, specifies nesting; | ||
- | | + | form is as above |
| | ||
+ | < | ||
RANDOM | RANDOM | ||
rtype | rtype | ||
+ | </ | ||
- | the RANDOM keyword occurs only if the current effect is random; | + | the RANDOM keyword occurs only if the current effect is random; |
- | | + | rtype is: |
- | | + | * ' |
+ | * ' | ||
+ | * ' | ||
+ | |||
+ | < | ||
OPTIONAL | OPTIONAL | ||
o1 o2.. oq | o1 o2.. oq | ||
+ | </ | ||
+ | causes extra effects appended to the animal effect; | ||
- | | + | current options include: |
- | | + | |
- | permanent environment | + | * ' |
- | + | * 'mpe' for maternal permanent environment; only if ' | |
+ | |||
+ | < | ||
FILE | FILE | ||
fped | fped | ||
+ | </ | ||
- | for animal and sire model only, fped specifies the pedigree file | + | for animal and sire model only |
+ | // | ||
+ | |||
+ | < | ||
+ | SKIP_HEADER | ||
+ | n | ||
+ | </ | ||
+ | This is optional. It skips the first n lines as header in the pedigree file. | ||
+ | |||
+ | < | ||
FILE_POS | FILE_POS | ||
an s d alt_dam yob | an s d alt_dam yob | ||
+ | </ | ||
- | for animal effect only; specifies positions in the pedigree file of animal | + | for animal effect only; |
- | an, sire s, dam d, alternate_dam | + | |
- | | + | specifies positions in the pedigree file of animal |
- | 1 2 3 0 0. If maternal effect is specified, the maternal effect is due to | + | |
- | position of d if alt_dam field is 0, or otherwise is due to alt_dam; | + | missing |
- | | + | |
- | dam. | + | if this line is not given, defaults are 1 2 3 0 0. |
+ | |||
+ | If maternal effect is specified, the maternal effect is due to position of d if alt_dam field is 0, or otherwise is due to alt_dam; | ||
+ | |||
+ | If alt_dam field is not zero, it should include ID of real or recipient dam. | ||
+ | < | ||
SNP_FILE | SNP_FILE | ||
fsnp | fsnp | ||
+ | </ | ||
- | optional; fsnp specifies files with ID and SNP information; | + | optional;\\ |
- | relationship matrix will be constructed as in Aguilar et al. (2010) and will | + | //fsnp// specifies files with ID and SNP information; |
- | | + | |
- | same format as fped and SNP info needs to start from a fixed column and | + | if present, the relationship matrix will be constructed as in Aguilar et al. (2010) and will |
- | | + | include the genomic information; |
- | at leats one space; see info for program PreGSf90 | + | |
- | | + | file //fsnp// should start with ID with the same format as fped and SNP info needs to start from a fixed column and |
+ | include digits 0, 1, 2 and 5; | ||
+ | |||
+ | |||
+ | ID and SNP info need to be separated by at leats one space; see info for program | ||
+ | |||
+ | < | ||
PED_DEPTH | PED_DEPTH | ||
p | p | ||
- | + | </ | |
- | | + | |
- | | + | optional |
- | | + | |
+ | for animal effect only; | ||
+ | |||
+ | //p// specifies the depth of pedigree search; | ||
+ | |||
+ | the default is 3 | ||
+ | |||
+ | all pedigrees are loaded if p=0. This is the fastest as it reads the pedigree file only once. However, if you want to extract the informative animals (genotyped and phenotyped animals + their ancestors traced back) put a large number like 100. With p=0, RENUMF90 tries to include all animals found in the raw pedigree file even if the animals in the pedigree are not related to the animals with phenotype or genotype. Thus, p=0 is not recommended unless your pedigree file is already prepared and consists in the informative animals or the animals of interest. | ||
+ | |||
+ | |||
+ | < | ||
GEN_INT | GEN_INT | ||
min avg max | min avg max | ||
+ | </ | ||
+ | optional | ||
- | | + | specifies minimum, average and maximum generation interval |
- | | + | |
- | | + | applicable only if year of birth present; minimum and maximum used for |
- | | + | pedigree checks |
- | | + | |
+ | average used to predict year of birth of parent with missing | ||
+ | pedigree. | ||
+ | |||
+ | < | ||
REC_SEX | REC_SEX | ||
i | i | ||
+ | </ | ||
+ | optional\\ | ||
+ | if only one sex has records, specifies which parent it is | ||
- | | + | used for pedigree checks. |
- | pedigree checks. | + | |
- | | + | < |
UPG_TYPE | UPG_TYPE | ||
t | t | ||
- | + | </ | |
- | | + | optional\\ |
- | | + | |
- | subsequent line should contain list of years to separate different UPG; | + | |
- | if t is ' | + | * if t is ' |
- | UPG number that this missing parent should be allocated to; in this option, | + | |
- | all known parents should have pedigree lines, i.e., each parent field should | + | |
- | contain either the ID of a real parent, or a negative UPG number. | + | |
- | if t is ' | + | * if t is ' |
- | custom_upg(year_of_birth, | + | |
- | + | * There are the other values for t: ' | |
+ | |||
+ | < | ||
+ | INBREEDING | ||
+ | inb_type | ||
+ | </ | ||
+ | optional\\ | ||
+ | use of inbreeding coefficients to compute inb/upg code in the 4th column of the output pedigree file. Inbreeding calculation is a default in RENUMF90 ≥ v1.157, even if this keyword is not used. | ||
+ | |||
+ | // | ||
+ | * ' | ||
+ | * ' | ||
+ | * 'self x' - Calculates inbreeding with selfing, where x is the column in the pedigree file with the number of selfing generation | ||
+ | * ' | ||
+ | |||
+ | < | ||
+ | FIXED_REGRESSION | ||
+ | r_type | ||
+ | </ | ||
+ | It is the same as '' | ||
+ | |||
+ | < | ||
RANDOM_REGRESSION | RANDOM_REGRESSION | ||
r_type | r_type | ||
+ | </ | ||
+ | Specifies that random regressions should be applied to the animal and | ||
+ | corresponding random effects (mat, pe and mpe) or the diagonal random effect. | ||
- | | + | this keyword also could be applied to set covariables for fixed effects; |
- | | + | |
- | applied to set covariables for fixed effects; r_type | + | //r_type// could be: |
- | random regressions are in the data, or " | + | * ' |
- | to be generated from a single data variable; | + | * " |
- | | + | |
+ | < | ||
RR_POSITION | RR_POSITION | ||
r1 .. rq | r1 .. rq | ||
- | for random regressions, | + | </ |
- | r_type=' | + | for random regressions, |
- | covariable if r_type=' | + | |
- | | + | * //r1-rq// specifies positions of covariables if r_type=' |
+ | * //r1// is order of legendre polynomial and //r2// is position of covariable if r_type=' | ||
+ | |||
+ | < | ||
(CO)VARIANCES | (CO)VARIANCES | ||
g | g | ||
+ | </ | ||
- | g are (co)variances for the animal effect; the dimensions of g should | + | //g// are (co)variances for the animal effect |
- | account for the maternal | + | |
- | | + | the dimensions of g should account for random correlated |
+ | |||
+ | < | ||
(CO)VARIANCES_PE | (CO)VARIANCES_PE | ||
gpe | gpe | ||
+ | </ | ||
+ | //gpe// are (co)variances for the PE effect if present | ||
- | gpe are (co)variances for the PE effect if present | + | < |
(CO)VARIANCES_MPE | (CO)VARIANCES_MPE | ||
gmpe | gmpe | ||
+ | </ | ||
+ | //gmpe// are (co)variances for the MPE effect if present | ||
- | gmpe are (co)variances for the MPE effect if present | ||
+ | ==== User-defined UPG code ==== | ||
+ | |||
+ | See [[undoc: | ||
+ | |||
+ | The program accepts one of the following keywords in '' | ||
+ | With one of these options, the program looks at a particular column in the pedigree file as a group code and use it for assigning the UPG code. If an animal has a missing parent, the program assigns a UPG code based on the group code. | ||
+ | |||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | For '' | ||
+ | |||
+ | FILE_POS | ||
+ | 1 2 3 0 0 5 | ||
+ | |||
+ | For '' | ||
+ | |||
+ | FILE_POS | ||
+ | 1 2 3 0 0 5 6 | ||
+ | |||
+ | The program now accept 3, 5, 6, or 7 items in '' | ||
+ | |||
+ | =====Extra comments===== | ||
Sections starting from EFFECTS can be repeated any number of types. | Sections starting from EFFECTS can be repeated any number of types. | ||
+ | |||
If (Co)variances for any effect are missing, they are substituted with matrices containing 1.0 | If (Co)variances for any effect are missing, they are substituted with matrices containing 1.0 | ||
on diagonals and 0.1 on off-diagonals. | on diagonals and 0.1 on off-diagonals. | ||
Line 204: | Line 321: | ||
The sequence of keywords should be as above although optional fields can be | The sequence of keywords should be as above although optional fields can be | ||
- | skipped. Keywords out of order may not be recognized. | + | skipped. |
+ | Keywords out of order may not be recognized. | ||
+ | |||
+ | =====Options===== | ||
The following options can added at the end of the parameter file to redefine | The following options can added at the end of the parameter file to redefine | ||
- | parameter | + | parameters |
- | - the default size of character fields | + | - the default size of character fields |
- | OPTION | + | < |
- | | + | OPTION |
+ | </ | ||
+ | where //nn// is the new size. | ||
- | - the size of th record length | + | - the size of th record length |
- | OPTION max_string_readline nn | + | < |
- | | + | OPTION max_string_readline nn |
+ | </ | ||
+ | where //nn// is the new size. | ||
- | - the maximun | + | - the maximum |
- | OPTION max_field_readline nn | + | < |
- | | + | OPTION max_field_readline nn |
+ | </ | ||
+ | where //nn// is the number of fields. | ||
+ | < | ||
+ | OPTION no_basic_statistics | ||
+ | </ | ||
+ | avoids the computation of basic statistics (min, max, correlations...) which take a certain time for very large data file. | ||
+ | |||
+ | < | ||
+ | OPTION inbreeding_method n | ||
+ | </ | ||
+ | |||
+ | allows choosing a method for inbreeding calculation. The inbreeding coefficients are used later (in the other programs) to set up the coefficients for the A-inverse. Existing options are: | ||
+ | |||
+ | * 7: Recursive tabular with self breeding generations . This method is for populations with selfing, i.e. wheat. | ||
+ | * 6: Hybrid parallel computing which is basically a parallel (OMP) version of Meuwissen and Luo (1992) | ||
+ | * 5: method of Tier (1990) | ||
+ | * 4: recursive tabular method | ||
+ | * 3: Modified Colleau by Sargolzaei et al. (2005) | ||
+ | * 2: Modified Meuwissen & Luo by Sargolzaei & Iwaisaki (2004) | ||
+ | * 1: Meuwissen and Luo (1992) | ||
+ | |||
+ | the default is method 1, and speed-ups are made using method 6 but this requires using several threads (e.g. using '' | ||
- | The end of the parameter file for RENUMF90 can contain many lines beginning with OPTION. | + | The end of the parameter file for '' |
- | All of these lines are passed to parameter file renf90.par to be used by | + | |
- | application programs. | + | |
+ | All of these lines are passed to parameter file renf90.par to be used by application programs. | ||
- | Combining fields or interactions | + | |
- | ================================ | + | =====Combining fields or interactions====== |
- | Several fields in the data file can be combined into one using a COMBINE keyword. | + | Several fields in the data file can be combined into one using a '' |
- | COMBINE a b c .... | + | < |
- | catenates b c ... into c. Keywords | + | COMBINE a b c .... |
- | file, but possibly after comments. There may be many combined fields. | + | </ |
+ | catenates | ||
+ | Keyword | ||
+ | |||
+ | There may be many combined fields. | ||
For example: | For example: | ||
- | COMBINE 7 2 3 4 | + | < |
- | combines content of fields 2 3 4 into field 7; the data file is not changed, | + | COMBINE 7 2 3 4 |
- | only the program treats field 7 as fields 2 3 4 put together (without spaces). | + | </ |
- | The combined fields can be treated as " | + | |
- | and if their total length is <9. Otherwise, they need to be used as " | + | combines content of fields 2 3 4 into field 7; |
- | Please note that the maximum size of the combined variable is limited by the | + | |
- | largest size of the " | + | the data file is not changed, only the program treats field 7 as fields 2 3 4 put together (without spaces). |
+ | |||
+ | |||
+ | The combined fields can be treated as " | ||
+ | |||
+ | Please note that the maximum size of the combined variable is limited by the largest size of the " | ||
- | Additive Pedigree File | + | =====Additive Pedigree File====== |
- | ====================== | + | |
The additive pedigree file(s) renadd* has the following structure: | The additive pedigree file(s) renadd* has the following structure: | ||
Line 253: | Line 406: | ||
4) 3 minus number of known parents | 4) 3 minus number of known parents | ||
5) known or estimated year of birth (0 if not provided) | 5) known or estimated year of birth (0 if not provided) | ||
- | 6) number of known parents (parents might be eliminated if not | + | 6) number of known parents (parents might be eliminated if not contributing; |
- | | + | |
7) number of records | 7) number of records | ||
- | 8) number of progenies | + | 8) number of progeny |
- | | + | 9) number of progeny |
- | 9) number of progenies | + | |
- | | + | |
10) original animal id | 10) original animal id | ||
- | Extensions | + | =====Extensions======= |
- | ========== | + | |
The program is being modified to support inbreeding, dominance, random | The program is being modified to support inbreeding, dominance, random | ||
Line 271: | Line 421: | ||
- | Example | + | =====Example====== |
- | ======== | + | |
- | data file - data.test | + | |
- | --------------------- | + | **__data file__** |
+ | |||
+ | < | ||
1 aa 34.5 11 12 zz | 1 aa 34.5 11 12 zz | ||
3 bb 21.333 22 23 xx | 3 bb 21.333 22 23 xx | ||
Line 282: | Line 433: | ||
3 aa 30 55 56 yy | 3 aa 30 55 56 yy | ||
5 bb 1234567.890 66 67 zz | 5 bb 1234567.890 66 67 zz | ||
+ | </ | ||
- | pedigree file - test.ped | + | |
- | ------------------------ | + | **__pedigree file__** |
+ | < | ||
qq 0 0 | qq 0 0 | ||
aa 0 0 | aa 0 0 | ||
Line 290: | Line 443: | ||
cc qq 0 | cc qq 0 | ||
dd 0 aa | dd 0 aa | ||
+ | </ | ||
- | parameter file - testpar1 | + | |
- | ------------------------- | + | **__parameter file__** |
+ | < | ||
# Parameter file for program renf90; it is translated to parameter | # Parameter file for program renf90; it is translated to parameter | ||
- | # file for BLUPF90 family | + | # file for BLUPF90 family programs. |
DATAFILE | DATAFILE | ||
data.test | data.test | ||
Line 335: | Line 490: | ||
RANDOM | RANDOM | ||
diagonal | diagonal | ||
+ | </ | ||
- | + | **__printout__** \\ | |
- | printout | + | \\ |
- | ------------------------------------------------------ | + | (temporary; the amount of details may change) |
+ | < | ||
| | ||
name of parameter file? testpar1 | name of parameter file? testpar1 | ||
Line 427: | Line 584: | ||
Wrote parameter file " | Wrote parameter file " | ||
Wrote renumbered data " | Wrote renumbered data " | ||
+ | </ | ||
- | + | **__new | |
- | new parameter | + | < |
- | ------------------------------- | + | |
# BLUPF90 parameter file created by RENF90 | # BLUPF90 parameter file created by RENF90 | ||
DATAFILE | DATAFILE | ||
Line 491: | Line 648: | ||
| | ||
0.1000 | 0.1000 | ||
+ | </ | ||
- | data file - renf90.dat | + | **__data file__** |
- | ---------------------- | + | < |
34.5 11 1 3 5 12 1 3 aa 1 | 34.5 11 1 3 5 12 1 3 aa 1 | ||
| | ||
Line 502: | Line 660: | ||
30 55 2 3 5 56 2 2 aa 3 | 30 55 2 3 5 56 2 2 aa 3 | ||
| | ||
+ | </ | ||
- | | + | |
- | -------------------------------------------------------- | + | **__Pedigree file__** |
+ | < | ||
1 6 3 1 0 2 2 0 0 bb | 1 6 3 1 0 2 2 0 0 bb | ||
6 0 0 1 0 0 0 2 0 qq | 6 0 0 1 0 0 0 2 0 qq | ||
Line 513: | Line 672: | ||
3 0 5 1 0 1 2 0 2 aa | 3 0 5 1 0 1 2 0 2 aa | ||
4 6 7 1 0 2 1 0 0 cc | 4 6 7 1 0 2 1 0 0 cc | ||
+ | </ | ||
- | renumbering tables - renf90.tables | + | **'' |
- | ---------------------------------- | + | < |
| | ||
| |
readme.renumf90.txt · Last modified: 2024/05/22 18:53 by dani