Oftentimes we need to run loops that take a long time, and we might like to see some kind of output indicating how far along through the process we are, or even how long we have to wait until the process is finished.

tictoc and beepr

When dealing with pieces of code that take a long time to run, I always use the packages tictoc and beepr.

library(tidyverse)
library(tictoc)
library(beepr)

tictoc is a simple package for showing how long a piece of code took to run after it has finished. To use it you put tic() at the beginning and toc() at the end. beepr is a package with a function beep() which makes a ‘ding’ noise. I put this at the end of a chunk of code so that I get an alert that it has finished.

Here is a simple implementation. The function Sys.sleep just puts R to rest for the time shown by the argument, which is one hundredth of a second in the example below. Since it loops over this 500 times it should take approximately 5 seconds.

tic()
for(i in 1:500){
  Sys.sleep(0.01)
}
toc()
beep()
## 7.82 sec elapsed

There are 11 different sounds possible with the function beep(). You can also include an expression to test before making the sound. You could think about a process where different noises are made depending on the outcome. Have a look at the help for this function using ?beep to learn more.

Outputting a counter

You’re probably already aware that you can use print to print some increment variable to the console to keep track of a loop. A nice variation on this is to use cat instead, and then include cat("\r") afterwards. The effect of this is that all of the output stays on the same line, so the console does not fill up with progress text. Try the following:

for(i in 1:6){
  cat("The value of i is:", i)
  cat("\r")
  
  Sys.sleep(1)
}
## Sample output:
## The value of i is: 4

You could have included the "\r" with the initial cat expression, i.e. cat("The value of i is:", i , "\r"). To be best of my knowledge, it is not possible to have two lines of progress output in R.

Suppose you had hundreds or thousands of increments, and you only wanted show one in every ten or twenty in the console. You could include an if loop for a subset of the increments like so (one in every hundred steps is printed to the console here):

for(i in 1:1000){
  Sys.sleep(0.01)
  
  if(i %in% seq(0,1000, by=100)){
    cat("The value of i is:", i , "\r")
  }
  
}
## Sample output:
## The value of i is: 700

Of course you can print the name of a file or dataset or whatever is being processed too. You could try printing a percent like so:

for(i in 1:6){
  cat("The value of i is", i , "and we are at" , floor(i*100/6) , "% \r")
  
  Sys.sleep(1)
}
## Sample output:
## The value of i is 2 and we are at 33 %

Progress bars

Within base R there is a neat functionality for outputting a progress bar to the console. You have to set it up before the loop using txtProgressBar, and then update its output within the loop using setTxtProgressBar. The first two arguments of txtProgressBar are the start and end increments. There are a number of other optional variables, the only one of which I would bother with is ‘style’, which I always set to 3 as it shows the percentage completed to the right. Then the function setTxtProgressBar takes the progress bar object that you’ve created as its first argument, and the increment as the second argument.

my_progress_bar <- txtProgressBar(1,500, style = 3)

for(i in 1:500){
  setTxtProgressBar(my_progress_bar , i)
  Sys.sleep(0.01)
}
## Sample progress bar output:
## 
  |                                                                            
  |==========================================                            |  60%

There is another option for a popout window using analgous functions winProgressBar and setWinProgressBar (for Windows machines), again available within base R. A nice feature of this is that it allows title and label arguments, which you could use to describe the process in more detail. For example you could use label to say what file is being processed.

Remaining time

Maybe you’d like to know how long is left before your code is finished, or what time you can expect your code to be finished. Of course this depends on having a sequence of tasks which take a uniform amount of time to complete and consistent processing speed, but let’s say that’s roughly the case. On the first cycle of a loop we can store the start_time, and then for each subsequent cycle we can calculate the elapsed_time as the difference between the current time and start_time. We can estimate total_time by scaling elapsed time by the total number of steps versus the completed number of steps. From there we can output the remaining_time. The code for this is shown below.

for(i in 1:10){
  
  if(i==1){start_time <- Sys.time()}
  else{
    elapsed_time <- Sys.time() - start_time
    # Note that the 10 here must match the number of steps 
    total_time <- elapsed_time*10/i
    remaining_time <- round(total_time-elapsed_time, digits=2)
    
    Sys.sleep(1.5)
    
    cat(paste("Remaining time:", remaining_time , "seconds"))
    cat("\r")
  }
  
}
## Sample output:
## Remaining time: 5.26 seconds

We can create a variation that outputs the time in minutes or that specifies the clock time when the loop will be finished. Below we use the lubridate package to add the remaining time (which is in seconds) to the clock time. It is formatted using "%r", which gives the time in hours, minutes and seconds.

for(i in 1:20){
  
  if(i==1){start_time <- Sys.time()}
  else{
    elapsed_time <- Sys.time() - start_time
    # Note that the 20 here must match the number of steps 
    total_time <- elapsed_time*20/i
    remaining_time <- round(total_time-elapsed_time, digits=2)
    finish_time <- Sys.time() + lubridate::seconds(remaining_time) 
    
    Sys.sleep(1.5)
    
    cat(paste("Current time is", format(Sys.time() , "%r") , 
              "and process will be finished at" , format(finish_time , "%r")))
    cat("\r")
  }
  
}
## Sample output:
## Current time is 10:41:54  and process will be finished at 10:58:34

A custom progress bar

This is a pretty silly/complicated example, but let’s say we wanted our own custom progress bar in the console with a small piece of text in the middle. Here’s a way of doing that. I set the full width of the bar, full_bar, equal to the width of the console using the handy function console_width from the cli package. Then the width of the bar at any given moment (bar_partial) is calculated as the proportion of full_bar using the increment i. The unfinished, blank part of the bar called space_width is defined as the difference between full_bar and bar_partial. To make the bar I use a function that I’ve defined called paste_collapse, which collapses a vector of characters into a single string. The message to be inserted is created and substituted into the middle of the bar as shown below. Here the message is simply the percentage completed.

paste_collapse <- function(...){
  paste0(... ,collapse="")
}

for(i in 1:15){
  
  full_bar = cli::console_width()
  bar_partial <- floor(i*full_bar/15)
  space_width <- full_bar-bar_partial
  # Make the bar as a series of '#' followed by a series of spaces. 
  bar <- paste_collapse(paste_collapse(rep("=",bar_partial)) , paste_collapse(rep(" ",space_width)))
  # Add a vertical line to each end of the bar
  bar <- paste_collapse("|", bar, "|")
  
  # create your own message here. Don't make it too long!
  message <- paste0( round(i*100/15), "% complete")
  # pop a space on either side of the message.
  message <- paste0(" ", message , " ")
  # the message should be located at the midpoint of the bar
  where_to_put_message <- floor((full_bar - str_length(message))*0.5)
  # sub the message into the bar
  str_sub(bar , where_to_put_message , where_to_put_message + str_length(message) ) <- message
  
  Sys.sleep(0.4)
  
  cat(bar , "\r")
  rm(bar , full_bar , bar_partial, space_width, message)
}
## Sample output:
## |############################### 75% complete ##############                    |