User Tools

Site Tools


how_to_continue_gibbs_sampling_from_an_accidental_stop

Recovering from an accidental stop in Gibbs sampling

Overview

Quick answer

Our Gibbs sampling programs can continue the sampling only if you have prepared something for the accidental stop before running the program. Otherwise, please run it again.

There are two options:

  • If you are going to use THRGIBBS1F90, use OPTION save_halfway_samples.
  • Otherwise, use a shell script shown in below.

Details

The idea is to save the current samples of “solutions” and variance components. By default, the samples of variance components will be saved in gibbs_samples but the samples of “solutions” are saved only in the last round of sampling. To continue the sampling, the program needs both gibbs_samples and last_solutions.

The option save_halfway_samples THRGIBBS1F90 saves last_solutions in every *n* rounds. If the option is not available, you can perform small sampling repeatedly. For example, you can get 1000 samples in one run to get “last_solutions” and continue the next run using the previous results. A shell script can automate this process. It is not elegant but it will work.

Methods

In both methods, the continued sampling may give you slightly different results from the non-breaking run. This is because of numerical error. The saved samples of variance components have limited precision and it creates errors in subsequent sampling. In general, this error is trivial and you can ignore it.

Option for THRGIBBS1F90

See THRGIBBS1F90 for details.

Shell script for any Gibbs samplers

Modify the following shell script according to your analysis and run it with bash (terminal in Linux and macOS). Before running the script, make sure this script works in your environment. Please read the comment lines in the script to continue the sampling.

run_gibbs.sh
#!/bin/bash
#
# A suggested script to save last_solutions in every N rounds.
# Please change the variables according to your analysis.
#
# To continue the sampling, 1) rename last_run.* to the
# original file i.e.
#    last_run.gibbs_samples          -> gubbs_samples
#    last_run.last_solutions         -> last_solutions
#    last_run.fort.99                -> fort.99
#    last_run.binary_final_solutions -> binary_final_solutions
#
# and 2) put the number of previous samples to the variable 
# "previous_samples" below.
#
 
# This is 0 for the first time.
# Put appropriate value if you continue the sampling.
previous_samples=0
 
# Put your favorite value here.
total_samples=100000
 
# Save the intermediate samples every this rounds.
save_rounds=10000
 
# set your parameter file
parameter_file=renf90.par
 
# thinning: samples saved in every n rounds
thinning=1
 
# program name
program=thrgibbs1f90
 
# run the program
for previous in `seq $previous_samples $save_rounds $total_samples`; do
   # the current + last
   current_start=`expr $previous + 1`
   current_end=`expr $previous + $save_rounds`
   if [ $current_start -gt $total_samples ]; then
      break
   fi
 
   # create a parameter file with "OPTION cont"
   parameter_file_cont=$parameter_file.cont
   if [ $previous -eq 0 ]; then
      cp $parameter_file $parameter_file_cont
   else
      cp $parameter_file $parameter_file_cont
      echo OPTION cont $previous >> $parameter_file_cont
   fi
 
   # input
   echo $parameter_file_cont > input.txt
   echo $save_rounds 0   >> input.txt
   echo $thinning        >> input.txt
   echo "#" Running from $current_start to $current_end >> input.txt
 
   # run the program
   $program < input.txt
   if [ $? -ne 0 ]; then
      break
   fi
 
   # backup the files
   echo 'The files last_run.* have the previous' $current_end 'samples.' > last_run.txt
   cp gibbs_samples          last_run.gibbs_samples
   cp last_solutions         last_run.last_solutions
   cp fort.99                last_run.fort.99
   cp binary_final_solutions last_run.binary_final_solutions
done
how_to_continue_gibbs_sampling_from_an_accidental_stop.txt · Last modified: 2018/05/19 22:05 by yutaka