======================================================= RENUMF90 - renumbering program for the BLUPF90 family ======================================================= Ignacy Misztal, University of Georgia August 27, 2001- November 29, 2004 Preliminary version Summary ======= RENUMF90 is a renumbering program for the BLUPF90 family of programs. It supports multiple traits, different effects per trait, alphanumeric and numeric fields. The program provides data statistics, performs comprehensive pedigree checking, and supports unknown parent groups etc. It accepts files where fields in data and pedigree files are separated by spaces. The program is still in active development so errors are possible and some features may not work or work incorrectly. Warnings - input files cannot contain character #. - missing animals have code 0; 00 may be treated as known animal Structure of parameter file =========================== The parameter file contains keywords in capital followed by specifications of a given effect/data item. The keywords need to be typed exactly. Specific keywords need to occur sequentially, as shown below. Bugs ==== IDs starting with "-" may not work Fields in the parameter file ============================= # Parameter file for program renf90; it is translated to parameter # file for BLUPF90 family of programs. Lines with # are treated as comments DATAFILE f1 The data file is f1 TRAITS t1 t2 .. tn t1-tn are positions of traits in datafile; n defines the number of traits FIELDS_PASSED TO OUTPUT p1 p2 .. pm fields p1-pn are passed to output without changes; can be empty WEIGHT(S) w w is position of weight if present; can be empty RESIDUAL_VARIANCE r r is matrix of residual (co)variances of size n x n EFFECT e1.. en type form this line defines one group of effects; e1 .. en are positions of this effect for all traits; positions can be different for each trait for fixed effects; for random effects, only one position + 0 (misising) efefct are possible. type is 'cross' for crossclassified or 'cov' for covariables form is 'alpha' for alphanumeric or 'numer' for numeric NESTED d1 .. dn form optional for covariables only, specifies nesting; form is as above RANDOM rtype the RANDOM keyword occurs only if the current effect is random; rtype is 'diagonal', 'sire' or 'animal' OPTIONAL o1 o2.. oq o1 cause extra effects appended to the animal effect; current options include 'pe' for permanent environment, 'mat' for maternal, and 'mped' for maternal permanent environment FILE fped for animal and sire model only, fped specifies the pedigree file FILE_POS an s d alt_dam yob for animal effect only; specifies positions in the pedigree file of animal an, sire s, dam d, alternate_dam dam rec_dam , and year of birth yob; missing alt_dam or yob can be replaced by 0; if this line is not given, defaults are 1 2 3 0 0. If maternal efefct is specified, the maternal effect is due to position of d if alt_dam field is 0, or otherwise is due to alt_dam; If alt_dam field is not zero, it should include ID of real or recipient dam. PED_DEPTH p optional for animal effect only; p specifies the depth of pedigree search; the default is 3; all pedigrees are loaded if p=0. GEN_INT min avg max optional; specifies minimum, average and maximum generation interval; applicable only if year of birth present; minimum and maximum used for pedigree checks; average used to predict year of birth of parent with missing pedigree. REC_SEX i optional; if only one sex has records, specifies which parent it is; used for pedigree checks. UPG_TYPE t optional; if t is 'yob', the asignment is based on year of birth; the subsequent line should contain list of years to separate different UPG; (CO)VARIANCES g g are (co)variances for the animal effect; the dimensions of g should account for the maternal effect if present (CO)VARIANCES_PE gpe gpe are (co)variances for the PE effect if present (CO)VARIANCES_MPE gmpe gmpe are (co)variances for the MPE effect if present Sections starting from EFFECTS can be repeated any number of types. If (Co)variances for any effect are missing, they are substituted with matrices containing 1.0 on diagonals and 0.1 on off-diagonals. Warning: for variance estimation by EM REML,usually there is improved convergence rate if the starting values for (co)variances are too large than too small. The sequence of keywords should be as above although optional fields can be skipped. Keywords out of order may not be recognized. Combining fields or interactions ================================ Several fields in the data file can be combined into one using a COMBINE keyword. COMBINE a b c .... catenates b c ... into c. Keywords COMBINE need to be on top of the parameter file, but possibly after comments. There may be many combined fields. For example: COMBINE 7 2 3 4 combines content of fields 2 3 4 into field 7; the data file is not changed, only the program treats field 7 as fields 2 3 4 put together (without spaces). The combined fields can be treated as "numeric", if they are composed of numbers and if their total length is <9. Otherwise, they need to be used as "alpha". Please note that the maximum size of the combined variable is limited by the largest size of the "alpha" field. Additive Pedigree File ====================== The additive pedigree file(s) renadd* has the following structure: 1) animal number (from 1) 2) parent 1 number or unknown parent group number for parent 1 3) parent 2 number or unknown parent group number for parent 2 4) 3 minus number of known parents 5) known or estimated year of birth (0 if not provided) 6) number of known parents (parents might be eliminated if not contributing 7) number of records 8) number of progenies (before elimination due to other effects) as parent 1 9) number of progenies (before elimination due to other effects) as parent 2 10) original animal id Extensions ========== The program is being modified to support inbreeding, dominance, random regressions with automatic calculations of Legendre polynomials Example ======== data file - data.test --------------------- 1 aa 34.5 11 12 3 bb 21.333 22 23 8 cc 23.666 33 34 1 dd 29 44 45 3 aa 30 55 56 5 bb 1234567.890 66 67 pedigree file - test.ped ------------------------ qq 0 0 aa 0 0 bb qq aa cc qq 0 dd 0 aa parameter file for simple additive model ------------------------- # Parameter file for program renf90; it is translated to parameter # file for BLUPF90 family f programs. # # This file is for a two-trait matrenal model DATAFILE data.test TRAITS 3 4 RESIDUAL_VARIANCE 5 2 2 4 EFFECT 1 2 cross alpha EFFECT 2 2 cross alpha RANDOM animal FILE test.ped (CO)VARIANCES 10 3 2 1 3 11 4 5 2 4 12 6 1 5 6 13.01 parameter file for two-trait maternal model - testpar1 ------------------------- # Parameter file for program renf90; it is translated to parameter # file for BLUPF90 family of programs. DATAFILE data.test TRAITS 3 4 FIELDS_PASSED TO OUTPUT 2 1 #passing alphanumeric WEIGHT(S) RESIDUAL_VARIANCE 5 2 2 4 EFFECT 1 2 cross alpha EFFECT 2 2 cross alpha RANDOM animal OPTIONAL mat mpe pe FILE test.ped (CO)VARIANCES 10 3 2 1 3 11 4 5 2 4 12 6 1 5 6 13.01 (CO)VARIANCES_PE 5.3 2.1 2.1 4.85 (CO)VARIANCES_MPE 1.03 .27 .27 .85 EFFECT 4 0 cov NESTED 1 0 alpha EFFECT 1 1 cross alpha RANDOM diagonal printout (temporary; the amount of details may change) ------------------------------------------------------ datafile:data.test traits: 3 4 fields passed: 2 1 R 5.0000 2.0000 2.0000 4.0000 ef= 1 item_kind=alpha ef= 2 item_kind=alpha Optional maternal effect Optional maternal permanent environment Optional permanent environment pedigree file name:test.ped positions of animal, sire, dam, recipient dam and yob 1 2 3 0 0 Reading (CO)VARIANCES: 4 x 4 Reading (CO)VARIANCES_PE: 2 x 2 Reading (CO)VARIANCES_MPE: 2 x 2 ef= 3 item_kind=alpha ef= 4 item_kind=alpha read parameter file hash tables for effects set up read 6 records Effect group 1 of column 1 with 4 levels Effect group 1 of column 2 with 4 levels Effect group 2 of column 1 with 4 levels Effect group 3 of column 1 with 4 levels Effect group 4 of column 1 with 4 levels wrote statistics in file renf90.tables random effect 2 type:animal opened output pedigree file: renadd02.ped read 5 pedigree records Number of animals with records: 4 Number of parents without records: 1 Number of phantom dams: 3 Total number of animals: 8 random effect 4 type:diag prepared parameter file wrote renumbered data new parameter file - renf90.par ------------------------------- # BLUPF90 parameter file created by RENF90 DATAFILE renf90.dat NUMBER_OF_TRAITS 2 NUMBER_OF_EFFECTS 7 OBSERVATION(S) 1 2 WEIGHT(S) EFFECTS: POSITIONS_IN_DATAFILE NUMBER_OF_LEVELS TYPE_OF_EFFECT[EFFECT NESTED] 3 4 4 cross 5 5 8 cross 6 6 8 cross 6 6 8 cross 5 5 8 cross 7 0 4 cov 8 0 9 9 4 cross RANDOM_RESIDUAL VALUES 5.00 2.00 2.00 4.00 RANDOM_GROUP 2 3 RANDOM_TYPE add_animal FILE renadd02.ped (CO)VARIANCES 10.0 3.00 2.00 1.00 3.00 11.0 4.00 5.00 2.00 4.00 12.0 6.00 1.00 5.00 6.00 13.0 RANDOM_GROUP 4 RANDOM_TYPE diagonal FILE (CO)VARIANCES 1.03 0.270 0.270 0.850 RANDOM_GROUP 5 RANDOM_TYPE diagonal FILE (CO)VARIANCES 5.30 2.10 2.10 4.85 RANDOM_GROUP 7 RANDOM_TYPE diagonal FILE (CO)VARIANCES 1.00 0.100 0.100 1.00