Benchmarking methods for reading text files in R (CC290)

Pat revisits his code for reading in FASTA-formatted DNA sequence files in R. First he takes on how to read in the sequence data. Then he removes a for loop. Finally, he revisits some of the functions from stringi to see if he can make further improvements in the performance of the code. Between all of the changes the function is now 3 times faster than it was before! He shows how to use scan, readLines, readr::read_lines, data.table::fread, and vroom::vroom_lines. This episode is part of an ongoing effort to develop an R package that implements the naive Bayesian classifier.

If you want to get a physical copy of R Packages: https://amzn.to/43pMR8L
If you want a free, online version of R packages: https://r-pkgs.org/

You can find my blog post for this episode at https://www.riffomonas.org/code_club/....

Check out the GitHub repository at the:
* Beginning of the episode: https://github.com/riffomonas/phyloty...
* End of the episode: https://github.com/riffomonas/phyloty...


#rstats #paste #paste0 #refactor #testthat #tdd #microbenchmark #vectors #rdp #16S #classification #classifier #microbialecology #microbiome

Support Riffomonas by becoming a Patreon member!
  / riffomonas  

Want more practice on the concepts covered in Code Club? You can sign up for my weekly newsletter at https://shop.riffomonas.org/youtube to get practice problems, tips, and insights.

If you're interested in purchasing a video workshop be sure to check out https://riffomonas.org/workshops/

You can also find complete tutorials for learning R with the tidyverse using...
Microbial ecology data: https://www.riffomonas.org/minimalR/
General data: https://www.riffomonas.org/generalR/



0:00 Introduction
7:47 Benchmarking reading in unformatted text data
15:57 Applying benchmarking results to read_fasta
16:52 Vectorizing creation of data frame
27:31 Further optimization of stringi functions
31:43 Revisiting vroom::vroom_lines
32:20 Importing packages and functions