Differences

This shows you the differences between two versions of the page.

--- readme.renumf90 [2012/05/29 11:31] – ignacio
+++ readme.renumf90 [2022/01/22 20:26] – [Fields in the parameter file] dani
@@ Line 1: / Line 1: @@
 =======RENUMF90======
-renumbering program for the BLUPF90 family now works with SNP info\\
+A renumbering program for the ''BLUPF90'' family now works with SNP info\\
 Ignacy Misztal and Ignacio Aguilar, University of Georgia\\
 August 27, 2001 - Mar 17, 2011
@@ Line 7: / Line 7: @@
 =====Summary=====
-<file>
 RENUMF90 is a renumbering program for the BLUPF90 family of programs. It
 supports multiple traits, different effects per trait, alphanumeric and
@@ Line 18: / Line 18: @@
 Warnings
-	- input files cannot contain character #.
+  * input files cannot contain character #.
-	- missing animals have code 0; 00 may be treated as a known animal
+  * missing animals have code 0; 00 may be treated as a known animal
-Structure of parameter file
+=====Structure of parameter file======
-===========================
 The parameter file contains keywords in capital followed by specifications
-of a given effect/data item. The keywords need to be typed exactly. Specific
+of a given effect/data item.\\ The keywords need to be typed exactly.\\ Specific
 keywords need to occur sequentially, as shown below.
-Bugs
-====
+=====Bugs=====
 IDs starting with "-" may not work
-Fields in the parameter file
-=============================
-# Parameter file for program renf90; it is translated to parameter
+=====Fields in the parameter file=====
-# file for BLUPF90 family of programs.
-   Lines with # are treated as comments
+# Parameter file for renumf90. It is translated into a parameter file for the BLUPF90 family of programs.
+Lines with # are treated as comments
+<file>
 DATAFILE
 f1
+</file>
+The data file is f1
-   The data file is f1
+<file>
+SKIP_HEADER
+n
+</file>
+This is optional. It skips the first n lines as header in the data file.
+<file>
 TRAITS
 t1 t2 .. tn
+</file>
+t1-tn are positions of traits in datafile; n defines the number of traits
-   t1-tn are positions of traits in datafile; n defines the number of traits
+<file>
 FIELDS_PASSED TO OUTPUT
 p1 p2 .. pm
+</file>
-   fields p1-pn are passed to output without changes; can be empty
+fields p1-pn are passed to output without changes; can be empty
+<file>
 WEIGHT(S)
 w
-   w is  position of weight if present; can be empty
+</file>
+w is  position of weight if present; can be empty
+<file>
 RESIDUAL_VARIANCE
 r
-   r is matrix of residual (co)variances  of size n x n
+</file>
+r is matrix of residual (co)variances  of size n x n
+<file>
 EFFECT
 e1.. en type form
+</file>
+this line defines one group of effects; e1 .. en are positions of this effect for all traits;
-   this line defines one group of effects; e1 .. en are positions of this
+positions can be different for each trait for fixed effects;
-   effect for all traits; positions can be different for each trait for fixed
-   effects; for random effects, only one position + 0 (misising) efefct are
-   possible.
-   type is 'cross' for crossclassified or 'cov' for covariables
-   form is 'alpha' for alphanumeric or 'numer' for numeric
+for random effects, only one position + 0 (misising) efefct are
+possible.
+  * type is 'cross' for crossclassified or 'cov' for covariables
+  * form is 'alpha' for alphanumeric or 'numer' for numeric
+<file>
 NESTED
 d1 .. dn form
+</file>
+optional for covariables only, specifies nesting;
-   optional for covariables only, specifies nesting; form is as above
+form is as above
+<file>
 RANDOM
 rtype
+</file>
-   the RANDOM keyword occurs only if the current effect is random;
+the RANDOM keyword occurs only if the current effect is random;
-   rtype is 'diagonal', 'sire' or 'animal'
+rtype is:
+  * 'diagonal'
+  * 'sire'; not yet implemented
+  * 'animal'
+<file>
 OPTIONAL
 o1 o2.. oq
+</file>
+causes extra effects appended to the animal effect;
-   causes extra effects appended to the animal effect; current options include
+current options include:
-   'pe' for permanent environment, 'mat' for maternal, and 'mped' for maternal
+  * 'pe' for permanent environment
-   permanent environment
+  * 'mat' for maternal
+  * 'mpe' for maternal permanent environment; only if 'mat' is used
+<file>
 FILE
 fped
+</file>
-   for animal and sire model only, fped specifies the pedigree file
+for animal and sire model only
+ //fped// specifies the pedigree file
+<file>
+SKIP_HEADER
+n
+</file>
+This is optional. It skips the first n lines as header in the pedigree file.
+<file>
 FILE_POS
 an s d alt_dam yob
+</file>
+for animal effect only;
+specifies positions in the pedigree file of animal (an), sire (s), dam (d), alternate dam (alt_dam) , and year of birth (yob)
+missing alt_dam or yob can be replaced by 0
+if this line is not given, defaults are 1 2 3 0 0.
+If maternal effect is specified, the maternal effect is due to position of d if alt_dam field is 0, or otherwise is due to alt_dam;
-   for animal effect only; specifies positions in the pedigree file of animal
+If alt_dam field is not zero, it should include ID of real or recipient dam.
-   an, sire s, dam d, alternate_dam dam rec_dam , and year of birth yob; missing
-   alt_dam or yob can be replaced by 0; if this line is not given, defaults are
-2 3 0 0. If maternal effect is specified, the maternal effect is due to
-   position of d if alt_dam field is 0, or otherwise is due to alt_dam; If
-   alt_dam field is not zero, it should include ID of real or recipient
-   dam.
+<file>
 SNP_FILE
 fsnp
+</file>
-   optional; fsnp specifies files with ID and SNP information; if present, the
+optional;\\
-   relationship matrix will be constructed as in Aguilar et al. (2010) and will
+//fsnp// specifies files with ID and SNP information;
-   include the genomic information; file fsnp should start with ID with the
-   same format as fped and SNP info needs to start from a fixed column and
+if present, the relationship matrix will be constructed as in Aguilar et al. (2010) and will
-   include digits 0, 1, 2 and 5; ID and SNP info need to be separated by
+include the genomic information;
-   at leats one space; see info for program PreGSf90
+file //fsnp// should start with ID with the same format as fped and SNP info needs to start from a fixed column and
+include digits 0, 1, 2 and 5;
+ID and SNP info need to be separated by at leats one space; see info for program [[readme.pregsf90|PreGSf90]]
+<file>
 PED_DEPTH
 p
+</file>
-   optional for animal effect only; p specifies the depth of pedigree search;
-   the default is 3; all pedigrees are loaded if p=0.
+optional
+for animal effect only;
+//p// specifies the depth of pedigree search;
+the default is 3
+all pedigrees are loaded if p=0.
+<file>
 GEN_INT
 min avg max
+</file>
+optional
-   optional; specifies minimum, average and maximum generation interval;
+specifies minimum, average and maximum generation interval
-   applicable only if year of birth present; minimum and maximum used for
-   pedigree checks; average used to predict year of birth of parent with missing
+applicable only if year of birth present; minimum and maximum used for
-   pedigree.
+pedigree checks
+average used to predict year of birth of parent with missing
+pedigree.
+<file>
 REC_SEX
 i
+</file>
+optional\\
+if only one sex has records, specifies which parent it is
-   optional; if only one sex has records, specifies which parent it is; used for
+used for pedigree checks.
-   pedigree checks.
+<file>
 UPG_TYPE
 t
+</file>
-   optional;
+optional\\
-   if t is 'yob', the asignment is based on year of birth; the
+  * if t is 'yob', the assignment is based on year of birth; the subsequent line should contain list of years to separate different UPG;
-   subsequent line should contain list of years to separate different UPG;
-   if t is 'in_pedigrees', the value of a missing parent should be -x, where x is
+  * if t is 'in_pedigrees', the value of a missing parent should be -x, where x is UPG number that this missing parent should be allocated to; in this option, all known parents should have pedigree lines, i.e., each parent field should contain either the ID of a real parent, or a negative UPG number.
-   UPG number that this missing parent should be allocated to; in this option,
-   all known parents should have pedigree lines, i.e., each parent field should
-   contain either the ID of a real parent, or a negative UPG number.
-   if t is 'internal',  allocation is by a user-written function
+  * if t is 'internal',  allocation is by a user-written function custom_upg(year_of_birth,sex,ID, parent_code); not yet implemented.
-   custom_upg(year_of_birth,sex,ID, parent_code).
+  * There are the other values for t: 'group', 'group_unisex', and 'group_sex'. See below for details.
+<file>
+INBREEDING
+inb_type
+</file>
+optional\\
+use of inbreeding coefficients to compute inb/upg code in the 4th column of the output pedigree file
+//inb_type// could be:
+  * 'pedigree' - the program computes inbreeding coefficients with Meuwissen and Luo (1992) using the pedigree to be saved in renaddxx.ped; calculated inbreeding coefficients will be saved in a file "renf90.inb" with the original ID
+  * 'file' - the program reads inbreeding coefficients from an external file. You should put the filename after 'file' e.g. 'file inbreeding.txt'. The file has at least 2 columns: original_ID and inbreeding value (from 0.0 to 1.0). The program just skips unnecessary IDs
+<file>
+FIXED_REGRESSION
+r_type
+</file>
+It is the same as ''RANDOM_REGRESSION'' (see the explanation below) but it is effective only for fixed effects.
+<file>
 RANDOM_REGRESSION
 r_type
+</file>
+Specifies that random regressions should be applied to the animal and
+corresponding random effects (mat, pe and mpe) or the diagonal random effect.
-   Specifies that random regressions should be applied to the animal and
+this keyword also could be applied to set covariables for fixed effects;
-   corresponding effects (mat, pe and mpe), this keyword also could be
-   applied to set covariables for fixed effects; r_type is 'data' if covariables for
+//r_type// could be:
-   random regressions are in the data, or "legendre' if legendre plynomials are
+  * 'data' if covariables for random regressions are in the data
-   to be generated from a single data variable; not yet implemented
+  * "legendre' if legendre plynomials are to be generated from a single data variable; not yet implemented
+<file>
 RR_POSITION
 r1 .. rq
-   for random regressions, r1-rq specifies positions of covariables if
+</file>
-   r_type='data', or r1 is order of legendre polynomial and r2 is position of
+for random regressions,
-   covariable if r_type='legendre'; not yet implemented
+  * //r1-rq// specifies positions of covariables if r_type='data'
+  * //r1// is order of legendre polynomial and //r2// is position of covariable if r_type='legendre'; not yet implemented
+<file>
 (CO)VARIANCES
 g
+</file>
-   g are (co)variances for the animal effect; the dimensions of g should
+//g// are (co)variances for the animal effect
-   account for the maternal effect if present
+the dimensions of g should account for random correlated effect if present (maternal or random regression)
+<file>
 (CO)VARIANCES_PE
 gpe
+</file>
+//gpe// are (co)variances for the PE effect if present
-   gpe are (co)variances for the PE effect if present
+<file>
 (CO)VARIANCES_MPE
 gmpe
+</file>
+//gmpe// are (co)variances for the MPE effect if present
-   gmpe are (co)variances for the MPE effect if present
+==== User-defined UPG code ====
+See [[undoc:renumf90|a separate documentation]] for details.
+The program accepts one of the following keywords in ''UPG_TYPE'': ''group'', ''group_unisex'', and ''group_sex''.
+With one of these options, the program looks at a particular column in the pedigree file as a group code and use it for assigning the UPG code. If an animal has a missing parent, the program assigns a UPG code based on the group code.
+  * ''group_unisex'': The program assigns a UPG code to the unknown parent regardless of the parent's sex.
+  * ''group'': The program assigns a separate UPG code to the unknown sire and dam.
+  * ''group_sex'': The user can specify a sex-specific UPG in the original pedigree file.
+For ''group_unisex'' and ''group'', the column in the pedigree file is specified with the 6th item in ''FILE_POS''. The following example tells the program the 5th column in the pedigree file as the group code. The group code will be treated as characters.
+    FILE_POS
+2 3 0 0 5
+For ''group_sex'', you need two additional columns in the pedigree file: one is for an unknown sire, and the other is for an unknown dam. For example, assume the 5th column is for unknown sire, and the 6th column is for unknown dam, the ''FILE_POS'' entry has 7 items.
+    FILE_POS
+2 3 0 0 5 6
+The program now accept 3, 5, 6, or 7 items in ''FILE_POS''.
+=====Extra comments=====
 Sections starting from EFFECTS can be repeated any number of types.
 If (Co)variances for any effect are missing, they are substituted with matrices containing 1.0
 on diagonals and 0.1 on off-diagonals.
@@ Line 201: / Line 316: @@
 The sequence of keywords should be as above although optional fields can be
-skipped. Keywords out of order may not be recognized.
+skipped.
+Keywords out of order may not be recognized.
+=====Options=====
 The following options can added at the end of the parameter file to redefine
-parameter used to read the input file:
+parameters used to read the input file:
- - the default size of character fields
+ - the default size of character fields (default = 20)
-	OPTION alpha size nn
+<file>
-   where nn is the new size.
+OPTION alpha_size nn
+</file>
+where //nn// is the new size.
- - the size of th record length
+ - the size of th record length (default = 800)
-	OPTION max_string_readline nn
+<file>
-   where nn is the new size.
+OPTION max_string_readline nn
+</file>
+where //nn// is the new size.
- - the maximun number of fields
+ - the maximun number of fields (default = 100)
-        OPTION max_field_readline nn
+<file>
-   where nn is the number of fields.
+OPTION max_field_readline nn
+</file>
+where //nn// is the number of fields.
-The end of the parameter file for RENUMF90 can contain many lines beginning with OPTION.
+The end of the parameter file for ''RENUMF90'' can contain many lines beginning with OPTION.
-All of these lines are passed to parameter file renf90.par to be used by
-application programs.
+All of these lines are passed to parameter file renf90.par to be used by application programs.
-Combining fields or interactions
-================================
+=====Combining fields or interactions======
-Several fields in the data file can be combined into one using a COMBINE keyword.
+Several fields in the data file can be combined into one using a ''COMBINE'' keyword.
-	COMBINE a b c ....
+<file>
-catenates b c ... into c. Keywords COMBINE need to be on top of the parameter
+COMBINE a b c ....
-file, but possibly after comments. There may be many combined fields.
+</file>
+catenates //b c// ... into //a//.\\
+Keyword COMBINE needs to be on top of the parameter file, but possibly after comments.
+There may be many combined fields.
 For example:
-	COMBINE 7 2 3 4
+<file>
-combines content of fields 2 3 4 into field 7; the data file is not changed,
+COMBINE 7 2 3 4
-only the program treats field 7 as fields 2 3 4 put together (without spaces).
+</file>
-The combined fields can be treated as "numeric", if they are composed of numbers
-and if their total length is <9. Otherwise, they need to be used as "alpha".
+combines content of fields 2 3 4 into field 7;
-Please note that the maximum size of the combined variable is limited by the
-largest size of the "alpha" field.
+the data file is not changed, only the program treats field 7 as fields 2 3 4 put together (without spaces).
+The combined fields can be treated as "numeric", if they are composed of numbers and if their total length is <9. Otherwise, they need to be used as "alpha".\\
+Please note that the maximum size of the combined variable is limited by the largest size of the "alpha" field.
-Additive Pedigree File
+=====Additive Pedigree File======
-======================
 The additive pedigree file(s) renadd* has the following structure:
@@ Line 250: / Line 381: @@
 ) 3 minus number of known parents
 ) known or estimated year of birth (0 if not provided)
-) number of known parents (parents might be eliminated if not
+) number of known parents (parents might be eliminated if not contributing;
-      contributing; if animal has genotype 10+number of know parents
+      if animal has genotype 10+number of know parents
 ) number of records
-) number of progenies (before elimination due to other effects)
+) number of progeny (before elimination due to other effects) as parent 1
-      as parent 1
+) number of progeny (before elimination due to other effects) as parent 2
-) number of progenies (before elimination due to other effects)
-      as parent 2
 ) original animal id
-Extensions
+=====Extensions=======
-==========
 The program is being modified to support inbreeding, dominance, random
@@ Line 268: / Line 396: @@
-Example
+=====Example======
-========
-data file - data.test
----------------------
+**__data file__** - ''data.test''
+<file>
 aa 34.5 11 12 zz
 bb 21.333 22 23 xx
@@ Line 279: / Line 408: @@
 aa  30 55 56 yy
 bb 1234567.890 66 67 zz
+</file>
-pedigree file - test.ped
-------------------------
+**__pedigree file__** - ''test.ped''
+<file>
 qq 0 0
 aa 0 0
@@ Line 287: / Line 418: @@
 cc qq 0
 dd 0 aa
+</file>
-parameter file - testpar1
--------------------------
+**__parameter file__** - ''testpar1''
+<file>
 # Parameter file for program renf90; it is translated to parameter
-# file for BLUPF90 family f programs.
+# file for BLUPF90 family programs.
 DATAFILE
 data.test
@@ Line 332: / Line 465: @@
 RANDOM
 diagonal
+</file>
+**__printout__** \\
-printout (temporary; the amount of details may change)
+\\
-------------------------------------------------------
+(temporary; the amount of details may change)
+<file>
  RENUMF90 version 1.93
  name of parameter file? testpar1
@@ Line 424: / Line 559: @@
  Wrote parameter file "renf90.par"
  Wrote renumbered data "renf90.dat"
+</file>
+**__new parameter file__** - ''renf90.par''
-new parameter file - renf90.par
+<file>
--------------------------------
 # BLUPF90 parameter file created by RENF90
 DATAFILE
@@ Line 488: / Line 623: @@
 .000      0.1000
 .1000       1.000
+</file>
-data file - renf90.dat
+**__data file__** - ''renf90.dat''
-----------------------
+<file>
 .5 11 1 3 5 12 1 3 aa 1
 .333 22 2 1 3 23 2 1 bb 3
@@ Line 499: / Line 635: @@
 55 2 3 5 56 2 2 aa 3
  1234567.890 66 3 1 3 67 3 3 bb 5
+</file>
- Pedigree file (same format as from renum) - renadd02.ped
- --------------------------------------------------------
+**__Pedigree file__** - ''renadd02.ped''
+<file>
 6 3 1 0 2 2 0 0 bb
 0 0 1 0 0 0 2 0 qq
@@ Line 510: / Line 647: @@
 0 5 1 0 1 2 0 2 aa
 6 7 1 0 2 1 0 0 cc
+</file>
-renumbering tables - renf90.tables
+**''renumbering tables''** - ''renf90.tables''
-----------------------------------
+<file>
  Effect group 1 of column 1 with 4 levels
  Value    #    consecutive number