readme.renumf90
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
readme.renumf90 [2012/05/29 11:42] – ignacio | readme.renumf90 [2024/05/15 18:40] – [Options] dani | ||
---|---|---|---|
Line 1: | Line 1: | ||
=======RENUMF90====== | =======RENUMF90====== | ||
- | renumbering program for the BLUPF90 family now works with SNP info\\ | + | A renumbering program for the '' |
Ignacy Misztal and Ignacio Aguilar, University of Georgia\\ | Ignacy Misztal and Ignacio Aguilar, University of Georgia\\ | ||
August 27, 2001 - Mar 17, 2011 | August 27, 2001 - Mar 17, 2011 | ||
Line 21: | Line 21: | ||
* missing animals have code 0; 00 may be treated as a known animal | * missing animals have code 0; 00 may be treated as a known animal | ||
+ | \\ | ||
+ | **Hint**: type '' | ||
=====Structure of parameter file====== | =====Structure of parameter file====== | ||
The parameter file contains keywords in capital followed by specifications | The parameter file contains keywords in capital followed by specifications | ||
- | of a given effect/data item. The keywords need to be typed exactly. Specific | + | of a given effect/data item.\\ The keywords need to be typed exactly.\\ Specific |
keywords need to occur sequentially, | keywords need to occur sequentially, | ||
Line 37: | Line 38: | ||
=====Fields in the parameter file===== | =====Fields in the parameter file===== | ||
- | # Parameter file for program renf90; it is translated | + | # Parameter file for renumf90. It is translated |
- | # file for BLUPF90 family of programs. | + | |
Lines with # are treated as comments | Lines with # are treated as comments | ||
| | ||
- | < | + | < |
DATAFILE | DATAFILE | ||
f1 | f1 | ||
Line 48: | Line 48: | ||
The data file is f1 | The data file is f1 | ||
- | < | + | < |
+ | SKIP_HEADER | ||
+ | n | ||
+ | </ | ||
+ | This is optional. It skips the first n lines as header in the data file. | ||
+ | |||
+ | < | ||
TRAITS | TRAITS | ||
t1 t2 .. tn | t1 t2 .. tn | ||
Line 57: | Line 63: | ||
FIELDS_PASSED TO OUTPUT | FIELDS_PASSED TO OUTPUT | ||
p1 p2 .. pm | p1 p2 .. pm | ||
- | < | + | </file> |
fields p1-pn are passed to output without changes; can be empty | fields p1-pn are passed to output without changes; can be empty | ||
< | < | ||
WEIGHT(S) | WEIGHT(S) | ||
- | w | + | w1 [w2 w3...wn] |
</ | </ | ||
- | w is | + | w1 [w2 w3...wn] are |
- | < | + | < |
RESIDUAL_VARIANCE | RESIDUAL_VARIANCE | ||
r | r | ||
Line 76: | Line 82: | ||
e1.. en type form | e1.. en type form | ||
</ | </ | ||
- | this line defines one group of effects; e1 .. en are positions of this | + | this line defines one group of effects; e1 .. en are positions of this effect for all traits; |
- | effect for all traits; positions can be different for each trait for fixed | + | |
- | effects; for random effects, only one position + 0 (misising) efefct are | + | positions can be different for each trait for fixed effects; |
+ | |||
+ | for random effects, only one position + 0 (missing) efefct are | ||
possible. | possible. | ||
* type is ' | * type is ' | ||
- | * form is ' | + | * for crossclassified effects: |
+ | * for covariables: | ||
< | < | ||
Line 87: | Line 96: | ||
d1 .. dn form | d1 .. dn form | ||
</ | </ | ||
- | optional for covariables only, specifies nesting; form is as above | + | optional for covariables only, specifies nesting; |
+ | |||
+ | form is as above | ||
| | ||
< | < | ||
Line 95: | Line 106: | ||
the RANDOM keyword occurs only if the current effect is random; | the RANDOM keyword occurs only if the current effect is random; | ||
- | rtype is ' | + | rtype is: |
+ | * ' | ||
+ | * ' | ||
+ | * ' | ||
- | < | + | < |
OPTIONAL | OPTIONAL | ||
o1 o2.. oq | o1 o2.. oq | ||
- | < | + | </file> |
- | causes extra effects appended to the animal effect; | + | causes extra effects appended to the animal effect; |
- | ' | + | |
- | permanent environment | + | |
- | < | + | current options include: |
+ | * ' | ||
+ | * ' | ||
+ | * ' | ||
+ | |||
+ | < | ||
FILE | FILE | ||
fped | fped | ||
</ | </ | ||
- | for animal and sire model only, fped specifies the pedigree file | + | for animal and sire model only |
+ | |||
+ | //fped// specifies the pedigree file | ||
+ | |||
+ | < | ||
+ | SKIP_HEADER | ||
+ | n | ||
+ | </ | ||
+ | This is optional. It skips the first n lines as header in the pedigree file. | ||
< | < | ||
Line 117: | Line 142: | ||
</ | </ | ||
- | for animal effect only; specifies positions in the pedigree file of animal | + | for animal effect only; |
- | an, sire s, dam d, alternate_dam | + | |
- | alt_dam or yob can be replaced by 0; if this line is not given, defaults are | + | specifies positions in the pedigree file of animal |
- | 1 2 3 0 0. If maternal effect is specified, the maternal effect is due to | + | |
- | position of d if alt_dam field is 0, or otherwise is due to alt_dam; | + | missing |
- | alt_dam field is not zero, it should include ID of real or recipient | + | |
- | dam. | + | if this line is not given, defaults are 1 2 3 0 0. |
+ | |||
+ | If maternal effect is specified, the maternal effect is due to position of d if alt_dam field is 0, or otherwise is due to alt_dam; | ||
+ | |||
+ | If alt_dam field is not zero, it should include ID of real or recipient dam. | ||
- | < | + | < |
SNP_FILE | SNP_FILE | ||
fsnp | fsnp | ||
</ | </ | ||
- | optional; fsnp specifies files with ID and SNP information; | + | optional;\\ |
- | relationship matrix will be constructed as in Aguilar et al. (2010) and will | + | //fsnp// specifies files with ID and SNP information; |
- | include the genomic information; | + | |
- | same format as fped and SNP info needs to start from a fixed column and | + | if present, the relationship matrix will be constructed as in Aguilar et al. (2010) and will |
- | include digits 0, 1, 2 and 5; ID and SNP info need to be separated by | + | include the genomic information; |
- | at leats one space; see info for program PreGSf90 | + | |
- | < | + | file //fsnp// should start with ID with the same format as fped and SNP info needs to start from a fixed column and |
+ | include digits 0, 1, 2 and 5; | ||
+ | |||
+ | |||
+ | ID and SNP info need to be separated by at leats one space; see info for program | ||
+ | |||
+ | < | ||
PED_DEPTH | PED_DEPTH | ||
p | p | ||
</ | </ | ||
- | optional | + | optional |
- | the default is 3; all pedigrees are loaded if p=0. | + | |
- | < | + | for animal effect only; |
+ | |||
+ | //p// specifies the depth of pedigree search; | ||
+ | |||
+ | the default is 3 | ||
+ | |||
+ | all pedigrees are loaded if p=0. This is the fastest as it reads the pedigree file only once. However, if you want to extract the informative animals (genotyped and phenotyped animals + their ancestors traced back) put a large number like 100. With p=0, RENUMF90 tries to include all animals found in the raw pedigree file even if the animals in the pedigree are not related to the animals with phenotype or genotype. Thus, p=0 is not recommended unless your pedigree file is already prepared and consists in the informative animals or the animals of interest. | ||
+ | |||
+ | |||
+ | < | ||
GEN_INT | GEN_INT | ||
min avg max | min avg max | ||
</ | </ | ||
- | optional; specifies minimum, average and maximum generation interval; | + | optional |
+ | |||
+ | specifies minimum, average and maximum generation interval | ||
applicable only if year of birth present; minimum and maximum used for | applicable only if year of birth present; minimum and maximum used for | ||
- | pedigree checks; average used to predict year of birth of parent with missing | + | pedigree checks |
+ | |||
+ | average used to predict year of birth of parent with missing | ||
pedigree. | pedigree. | ||
- | < | + | < |
REC_SEX | REC_SEX | ||
i | i | ||
</ | </ | ||
- | optional; if only one sex has records, specifies which parent it is; used for | + | optional\\ |
- | pedigree checks. | + | if only one sex has records, specifies which parent it is |
- | < | + | used for pedigree checks. |
+ | |||
+ | < | ||
UPG_TYPE | UPG_TYPE | ||
t | t | ||
</ | </ | ||
- | optional; | + | optional\\ |
- | if t is ' | + | |
- | subsequent line should contain list of years to separate different UPG; | + | |
- | if t is ' | + | * if t is ' |
- | UPG number that this missing parent should be allocated to; in this option, | + | |
- | all known parents should have pedigree lines, i.e., each parent field should | + | |
- | contain either the ID of a real parent, or a negative UPG number. | + | |
- | if t is ' | + | * if t is ' |
- | custom_upg(year_of_birth, | + | |
- | < | + | * There are the other values for t: ' |
+ | |||
+ | < | ||
+ | INBREEDING | ||
+ | inb_type | ||
+ | </ | ||
+ | optional\\ | ||
+ | use of inbreeding coefficients to compute inb/upg code in the 4th column of the output pedigree file. Inbreeding calculation is a default in RENUMF90 ≥ v1.157, even if this keyword is not used. | ||
+ | |||
+ | // | ||
+ | * ' | ||
+ | * ' | ||
+ | * 'self x' - Calculates inbreeding with selfing, where x is the column in the pedigree file with the number of selfing generation | ||
+ | * ' | ||
+ | |||
+ | < | ||
+ | FIXED_REGRESSION | ||
+ | r_type | ||
+ | </ | ||
+ | It is the same as '' | ||
+ | |||
+ | < | ||
RANDOM_REGRESSION | RANDOM_REGRESSION | ||
r_type | r_type | ||
</ | </ | ||
Specifies that random regressions should be applied to the animal and | Specifies that random regressions should be applied to the animal and | ||
- | corresponding effects (mat, pe and mpe), this keyword also could be | + | corresponding |
- | applied to set covariables for fixed effects; r_type is ' | + | |
- | random | + | |
- | to be generated from a single data variable; not yet implemented | + | |
- | < | + | this keyword also could be applied to set covariables for fixed effects; |
+ | |||
+ | //r_type// could be: | ||
+ | * ' | ||
+ | * " | ||
+ | |||
+ | < | ||
RR_POSITION | RR_POSITION | ||
r1 .. rq | r1 .. rq | ||
</ | </ | ||
- | for random regressions, | + | for random regressions, |
- | r_type=' | + | |
- | covariable if r_type=' | + | |
- | </ | + | |
- | < | + | * //r1-rq// specifies positions of covariables if r_type=' |
+ | * //r1// is order of legendre polynomial and //r2// is position of covariable if r_type=' | ||
+ | |||
+ | < | ||
(CO)VARIANCES | (CO)VARIANCES | ||
g | g | ||
</ | </ | ||
- | //g// are (co)variances for the animal effect; the dimensions of g should | + | //g// are (co)variances for the animal effect |
- | account for the maternal effect if present | + | |
- | < | + | the dimensions of g should account for random correlated effect if present (maternal or random regression) |
+ | |||
+ | < | ||
(CO)VARIANCES_PE | (CO)VARIANCES_PE | ||
gpe | gpe | ||
Line 209: | Line 279: | ||
//gpe// are (co)variances for the PE effect if present | //gpe// are (co)variances for the PE effect if present | ||
- | < | + | < |
(CO)VARIANCES_MPE | (CO)VARIANCES_MPE | ||
gmpe | gmpe | ||
Line 215: | Line 285: | ||
//gmpe// are (co)variances for the MPE effect if present | //gmpe// are (co)variances for the MPE effect if present | ||
+ | |||
+ | ==== User-defined UPG code ==== | ||
+ | |||
+ | See [[undoc: | ||
+ | |||
+ | The program accepts one of the following keywords in '' | ||
+ | With one of these options, the program looks at a particular column in the pedigree file as a group code and use it for assigning the UPG code. If an animal has a missing parent, the program assigns a UPG code based on the group code. | ||
+ | |||
+ | * '' | ||
+ | * '' | ||
+ | * '' | ||
+ | |||
+ | For '' | ||
+ | |||
+ | FILE_POS | ||
+ | 1 2 3 0 0 5 | ||
+ | |||
+ | For '' | ||
+ | |||
+ | FILE_POS | ||
+ | 1 2 3 0 0 5 6 | ||
+ | |||
+ | The program now accept 3, 5, 6, or 7 items in '' | ||
+ | |||
+ | =====Extra comments===== | ||
Sections starting from EFFECTS can be repeated any number of types. | Sections starting from EFFECTS can be repeated any number of types. | ||
+ | |||
If (Co)variances for any effect are missing, they are substituted with matrices containing 1.0 | If (Co)variances for any effect are missing, they are substituted with matrices containing 1.0 | ||
on diagonals and 0.1 on off-diagonals. | on diagonals and 0.1 on off-diagonals. | ||
Line 225: | Line 321: | ||
The sequence of keywords should be as above although optional fields can be | The sequence of keywords should be as above although optional fields can be | ||
- | skipped. Keywords out of order may not be recognized. | + | skipped. |
+ | Keywords out of order may not be recognized. | ||
+ | |||
+ | =====Options===== | ||
The following options can added at the end of the parameter file to redefine | The following options can added at the end of the parameter file to redefine | ||
- | parameter | + | parameters |
- | - the default size of character fields | + | - the default size of character fields |
- | OPTION | + | < |
- | | + | OPTION |
+ | </ | ||
+ | where //nn// is the new size. | ||
- | - the size of th record length | + | - the size of th record length |
- | OPTION max_string_readline nn | + | < |
- | | + | OPTION max_string_readline nn |
+ | </ | ||
+ | where //nn// is the new size. | ||
- | - the maximun | + | - the maximum |
- | OPTION max_field_readline nn | + | < |
- | | + | OPTION max_field_readline nn |
+ | </ | ||
+ | where //nn// is the number of fields. | ||
+ | < | ||
+ | OPTION missing x | ||
+ | </ | ||
+ | |||
+ | allows indicating that the missing value is the number x (e.g., 999), for instance, if 0 is a valid record. This is only to represent the missing value in the data. If there are covariables in the data, 0 is treated as a value, not missing information. Missing pedigree is always 0 and cannot be changed to another value. | ||
+ | |||
+ | < | ||
+ | OPTION missing_in_weights | ||
+ | </ | ||
+ | |||
+ | in addition, this indicates that if a weight for the trait is 0, then the value of the trait is converted to " | ||
+ | |||
+ | < | ||
+ | OPTION no_basic_statistics | ||
+ | </ | ||
+ | avoids the computation of basic statistics (min, max, correlations...) which take a certain time for very large data file. | ||
+ | |||
+ | < | ||
+ | OPTION inbreeding_method m | ||
+ | </ | ||
+ | |||
+ | allows choosing a method for inbreeding calculation. The inbreeding coefficients are used later (in the other programs) to set up the coefficients for the A-inverse. Existing values for m are: | ||
+ | | ||
+ | * 1: Meuwissen and Luo (1992) | ||
+ | * 2: Modified Meuwissen & Luo by Sargolzaei & Iwaisaki (2004) | ||
+ | * 3: Modified Colleau by Sargolzaei et al. (2005) | ||
+ | * 4: recursive tabular method | ||
+ | * 5: method of Tier (1990) | ||
+ | * 6: Hybrid parallel computing, which is basically a parallel (OMP) version of Meuwissen and Luo (1992) | ||
+ | * 7: Recursive tabular with self-breeding generations. For populations with selfing, i.e., wheat | ||
+ | |||
+ | The default is method 1. Large speed-ups are made using method 6, but this requires using several threads (e.g., using '' | ||
- | The end of the parameter file for RENUMF90 can contain many lines beginning with OPTION. | + | The end of the parameter file for '' |
- | All of these lines are passed to parameter file renf90.par to be used by | + | |
- | application programs. | + | |
+ | All of these lines are passed to the parameter file renf90.par to be used by application programs. | ||
+ | |||
+ | |||
+ | =====Combining fields or interactions====== | ||
+ | Several fields in the data file can be combined into one using a '' | ||
+ | < | ||
+ | COMBINE a b c .... | ||
+ | </ | ||
+ | catenates //b c// ... into //a//.\\ | ||
+ | Keyword COMBINE needs to be on top of the parameter file, but possibly after comments. | ||
- | Combining fields or interactions | + | There may be many combined fields. |
- | ================================ | + | |
- | Several fields in the data file can be combined into one using a COMBINE keyword. | + | |
- | COMBINE a b c .... | + | |
- | catenates b c ... into c. Keywords COMBINE need to be on top of the parameter | + | |
- | file, but possibly after comments. | + | |
For example: | For example: | ||
- | COMBINE 7 2 3 4 | + | < |
- | combines content of fields 2 3 4 into field 7; the data file is not changed, | + | COMBINE 7 2 3 4 |
- | only the program treats field 7 as fields 2 3 4 put together (without spaces). | + | </ |
- | The combined fields can be treated as " | + | |
- | and if their total length is <9. Otherwise, they need to be used as " | + | combines content of fields 2 3 4 into field 7; |
- | Please note that the maximum size of the combined variable is limited by the | + | |
- | largest size of the " | + | the data file is not changed, only the program treats field 7 as fields 2 3 4 put together (without spaces). |
+ | |||
+ | |||
+ | The combined fields can be treated as " | ||
+ | |||
+ | Please note that the maximum size of the combined variable is limited by the largest size of the " | ||
- | Additive Pedigree File | + | =====Additive Pedigree File====== |
- | ====================== | + | |
The additive pedigree file(s) renadd* has the following structure: | The additive pedigree file(s) renadd* has the following structure: | ||
Line 272: | Line 416: | ||
2) parent 1 number or unknown parent group number for parent 1 | 2) parent 1 number or unknown parent group number for parent 1 | ||
3) parent 2 number or unknown parent group number for parent 2 | 3) parent 2 number or unknown parent group number for parent 2 | ||
- | 4) 3 minus number of known parents | + | 4) 3 minus number of known parents |
5) known or estimated year of birth (0 if not provided) | 5) known or estimated year of birth (0 if not provided) | ||
- | 6) number of known parents (parents might be eliminated if not | + | 6) number of known parents (parents might be eliminated if not contributing; |
- | | + | |
7) number of records | 7) number of records | ||
- | 8) number of progenies | + | 8) number of progeny |
- | | + | 9) number of progeny |
- | 9) number of progenies | + | |
- | | + | |
10) original animal id | 10) original animal id | ||
- | Extensions | + | =====Extensions======= |
- | ========== | + | |
The program is being modified to support inbreeding, dominance, random | The program is being modified to support inbreeding, dominance, random | ||
Line 292: | Line 433: | ||
- | Example | + | =====Example====== |
- | ======== | + | |
- | data file - data.test | + | |
- | --------------------- | + | **__data file__** |
+ | |||
+ | < | ||
1 aa 34.5 11 12 zz | 1 aa 34.5 11 12 zz | ||
3 bb 21.333 22 23 xx | 3 bb 21.333 22 23 xx | ||
Line 303: | Line 445: | ||
3 aa 30 55 56 yy | 3 aa 30 55 56 yy | ||
5 bb 1234567.890 66 67 zz | 5 bb 1234567.890 66 67 zz | ||
+ | </ | ||
- | pedigree file - test.ped | + | |
- | ------------------------ | + | **__pedigree file__** |
+ | < | ||
qq 0 0 | qq 0 0 | ||
aa 0 0 | aa 0 0 | ||
Line 311: | Line 455: | ||
cc qq 0 | cc qq 0 | ||
dd 0 aa | dd 0 aa | ||
+ | </ | ||
- | parameter file - testpar1 | + | |
- | ------------------------- | + | **__parameter file__** |
+ | < | ||
# Parameter file for program renf90; it is translated to parameter | # Parameter file for program renf90; it is translated to parameter | ||
- | # file for BLUPF90 family | + | # file for BLUPF90 family programs. |
DATAFILE | DATAFILE | ||
data.test | data.test | ||
Line 356: | Line 502: | ||
RANDOM | RANDOM | ||
diagonal | diagonal | ||
+ | </ | ||
- | + | **__printout__** \\ | |
- | printout | + | \\ |
- | ------------------------------------------------------ | + | (temporary; the amount of details may change) |
+ | < | ||
| | ||
name of parameter file? testpar1 | name of parameter file? testpar1 | ||
Line 448: | Line 596: | ||
Wrote parameter file " | Wrote parameter file " | ||
Wrote renumbered data " | Wrote renumbered data " | ||
+ | </ | ||
- | + | **__new | |
- | new parameter | + | < |
- | ------------------------------- | + | |
# BLUPF90 parameter file created by RENF90 | # BLUPF90 parameter file created by RENF90 | ||
DATAFILE | DATAFILE | ||
Line 512: | Line 660: | ||
| | ||
0.1000 | 0.1000 | ||
+ | </ | ||
- | data file - renf90.dat | + | **__data file__** |
- | ---------------------- | + | < |
34.5 11 1 3 5 12 1 3 aa 1 | 34.5 11 1 3 5 12 1 3 aa 1 | ||
| | ||
Line 523: | Line 672: | ||
30 55 2 3 5 56 2 2 aa 3 | 30 55 2 3 5 56 2 2 aa 3 | ||
| | ||
+ | </ | ||
- | | + | |
- | -------------------------------------------------------- | + | **__Pedigree file__** |
+ | < | ||
1 6 3 1 0 2 2 0 0 bb | 1 6 3 1 0 2 2 0 0 bb | ||
6 0 0 1 0 0 0 2 0 qq | 6 0 0 1 0 0 0 2 0 qq | ||
Line 534: | Line 684: | ||
3 0 5 1 0 1 2 0 2 aa | 3 0 5 1 0 1 2 0 2 aa | ||
4 6 7 1 0 2 1 0 0 cc | 4 6 7 1 0 2 1 0 0 cc | ||
+ | </ | ||
- | renumbering tables - renf90.tables | + | **'' |
- | ---------------------------------- | + | < |
| | ||
| |
readme.renumf90.txt · Last modified: 2024/05/22 18:53 by dani