1. Parse

The .parse() function takes a string containing R code and returns a dataframe of its ‘syntax tree’. In other words, it breaks the string into ‘tokens’ that are recognised as units of R code: function calls, assignment arrows, etc.

I’ve used getParseData(parse()) to do the hard work of parsing the string into a dataframe with one row per token. The downside is that you must provide to it a file rather than a character object, so we first have to write it to a temporary file.

I’ve then filtered the dataframe to get only the tokens that are R code (i.e. they aren’t spaces) and renumbered the rows so they’re consecutive. This will be useful when we want to extract the function names for each set of parens.

.parse <- function(string) {
  
  file <- tempfile(fileext = ".R")
  writeLines(string, file)
  
  tokens <- getParseData(parse(file))
  parsed <- parsed[parsed$terminal == TRUE, ]
  rownames(parsed) <- as.character(seq(nrow(parsed)))
  
  return(parsed)
  
}

Here’s what the output looks like:

tree <- .parse(string)
tree[, c("line1", "col1", "parent", "token", "text")]

##    line1 col1 parent                token   text
## 1      1    1      3 SYMBOL_FUNCTION_CALL  first
## 2      1    6     40                  '('      (
## 3      1    7      6 SYMBOL_FUNCTION_CALL second
## 4      1   13     35                  '('      (
## 5      1   14      9 SYMBOL_FUNCTION_CALL  third
## 6      1   19     16                  '('      (
## 7      1   20     12            STR_CONST    'x'
## 8      1   23     16                  ')'      )
## 9      1   24     35                  ','      ,
## 10     1   26     23 SYMBOL_FUNCTION_CALL fourth
## 11     1   32     30                  '('      (
## 12     1   33     26            STR_CONST    'y'
## 13     1   36     30                  ')'      )
## 14     1   37     35                  ')'      )
## 15     1   38     40                  ')'      )

So each row is a recognised R token, e.g. the function name from the first() function is a SYMBOL_FUNCTION_CALL and 'x' is a STR_CONSTANT. Parentheses are recognised as separate special tokens too: '(' and ')'.

We also get returned the position of each token in the input (line* and col*) and a variable called parent which tells us something about the association of tokens. In our case, opening- and closing-parens have the same parent value.

2. Label

So we can tie our paren pairs together with the parent variable and we know where to place the biscuit with the line1 and col1 information. But how to extract the function name and ‘biscuitise’ it?

I’ve written the slightly awkward .label() function for this. It takes the output from .parse() and checks each row to see if it’s a closing-paren token; if so, it finds the position of the matching open-paren by parent; then it looks at the text of the token in the preceding row to get the function name and sticks that in a new column called label.

.label <- function(tree) {
  
  tree$label <- NA_character_
  
  for (tkn in seq_len(nrow(tree))) {
    
    tree$label[tkn] <- ifelse(
      tree$token[[tkn]] == "')'",
      tree[as.numeric(rownames(
        tree[tree$parent == tree$parent[[tkn]] & tree$token == "'('", ]
      )) - 1, "text"],
      NA_character_
    )
    
  }
  
  return(tree)
  
}

So now we have the required biscuit for each closing paren:

tree_lbl <- .label(tree)
tree_lbl[!is.na(tree_lbl$label), c("text", "label")]

##    text  label
## 8     )  third
## 13    ) fourth
## 14    ) second
## 15    )  first

3. Format

The last step needs involves sticking everything back together again. My quick solution is hacky and needs a refactor for sure.

The .format() function does a couple of awkward things: recognises and pastes commas to their preceding token (otherwise we’ll get lines in the output that start with a comma, which is valid, but not my style) and pastes in the biscuits with a suffixed # to mark it as a comment. Of course, this blocks downstream code, so we can add a linebreak with \n.

The output is still going to be a bit uggo though, so I employed {styler} to reformat it in tidyverse style. This is very definitely opinionated.

.format <- function(tree_lbl) {
  
  tree_lbl$comma <- c(
    ifelse(tree_lbl$text != ",", NA_character_, ",")[-1], NA_character_
  )  # lag commas
  
  tree_lbl <- tree_lbl[tree_lbl$token != "','", ]  # remove comma tokens
  
  tree_lbl$string <- NA_character_
  
  for (tkn in seq_len(nrow(tree_lbl))) {
    
    if (!is.na(tree_lbl$comma[tkn])) {  # when there's a comma
      
      if (!is.na(tree_lbl$label[tkn])) {  # paste with biscuit
        
        tree_lbl$string[tkn] <- paste0(
          "\n", tree_lbl$text[tkn], tree_lbl$comma[tkn],
          "#", tree_lbl$label[tkn], "\n"
        ) 
        
      } else if (is.na(tree_lbl$label[tkn])) {  # paste without biscuit
        
        tree_lbl$string[tkn] <- paste0(
          "\n", tree_lbl$text[tkn], tree_lbl$comma[tkn], "\n"
        ) 
        
      }
      
    } else if (is.na(tree_lbl$comma[tkn]) & !is.na(tree_lbl$label[tkn])) {
      
      tree_lbl$string[tkn] <- paste0(
        "\n", tree_lbl$text[tkn], "#", tree_lbl$label[tkn], "\n"
      ) 
      
    } else {  # no comma, no biscuit
      
      tree_lbl$string[tkn] <- tree_lbl$text[tkn]
      
    }
    
  }
  
  string_out <- paste0(tree_lbl$string, collapse = "")
  string_out <- gsub("\n\n", "\n", string_out)
  
  styled <- suppressWarnings(
    utils::capture.output(styler::style_text(string_out))
  )
  
  paste(styled, collapse = "\n")
  
}

Let’s hand over to .format() the labelled tree dataframe that was output from .label():

out <- .format(tree_lbl)
out

## [1] "first(\n  second(\n    third(\"x\"), # third\n    fourth(\"y\") # fourth\n  ) # second\n) # first"

So the output is a character vector, with one element per line of our output R file. You can see in the console how this looks.

cat(out)

## first(
##   second(
##     third("x"), # third
##     fourth("y") # fourth
##   ) # second
## ) # first

Real noice: we’ve got a comment after each closing bracket that notes which function it’s closing. You can argue that some of these biscuits are redundant, but the goal has been achieved!

A reminder of the original input:

string

## [1] "first(second(third('x'), fourth('y')))"

4. Combined function

And so, we can put these three steps together in one function: add_biscuits(), which is a cuter name than label_parens() or whatever.

add_biscuits <- function(string) { 
  .parse(string) |> .label() |> .format()
}

The output from each sub-function passes to the next, so it’s a nice chance to use the pipe operator (R >= v4.1).

Let’s try it on that awkward example with the sneaky extra bracket.

string2 <- "first(second(third(')')))"
cat(add_biscuits(string2))

## first(
##   second(
##     third(")") # third
##   ) # second
## ) # first

So only the ‘real’ closing-paren tokens have been recognised and labelled.