There is a major problem with the state of methodology of enrichment analysis with many articles lacking details about what was done, undermining the validitiy of the scientific results. Here we want to show how to use AI APIs to examine the methods in articles. This approach can be used to evaluate articles in the review phase.
We are using ChatGPT nano for this.
library(LLMR)
# get API key - keep it secret
OPENAI_API_KEY <- readLines("openai.key")[1]In the first test, we just ask a simple question.
# Create a configuration with more parameters
openai_cfg <- llm_config(
provider = "openai",
model = "gpt-4.1-nano",
api_key = OPENAI_API_KEY,
temperature = .5,
max_tokens = 250
)
# Ask a simple question
resp <- call_llm( openai_cfg, c(
system = "You are an expert data scientist. You always respond in terse
bullet lists.",
user = "When will you ever use OLS?"
),
json = TRUE
)
cat("GPT-4o-mini says:\n", resp, "\n")## GPT-4o-mini says:
## - Estimating linear relationships between variables
## - Predicting a continuous dependent variable
## - When assumptions of linearity, independence, homoscedasticity, and normality are reasonably met
## - As a baseline model for regression tasks
## - When interpretability of coefficients is important
## - For hypothesis testing on predictor significance
## - In econometrics, social sciences, and finance analysesIn this next example, we will load a text file and then ask ChatGPT about it. This allows us to run a checklist against articles.
# extract text
text_file_path = "PMC6350986.txt"
text_content <- readLines(text_file_path)
text_content <- paste(text_content, collapse = "\n")
# Prepare your question
question <- "In the attached article, extract the following informaton: (1) The bioinformatics
tools used for pathway enrichment analysis. (2) Whether the version number was provided.
(3) Whether p-value correction was used in functional enrichment analysis. (4) If
over-representation analysis was used, whether a background gene list was used."
# Create the prompt combining the text and the question
prompt <- paste(
"Here is a text:\n",
text_content,
"\nQuestion: ",
question,
"\nAnswer:"
)
response <- call_llm(config = openai_cfg , prompt, verbose = TRUE)## Full API Response:
## $id
## [1] "chatcmpl-C15uHABB29VriDj1pc6CRqrpq7Xtk"
##
## $object
## [1] "chat.completion"
##
## $created
## [1] 1754377069
##
## $model
## [1] "gpt-4.1-nano-2025-04-14"
##
## $choices
## $choices[[1]]
## $choices[[1]]$index
## [1] 0
##
## $choices[[1]]$message
## $choices[[1]]$message$role
## [1] "assistant"
##
## $choices[[1]]$message$content
## [1] "(1) The bioinformatics tool used for pathway enrichment analysis was Gene Set Enrichment Analysis (GSEA). \n(2) The version number of the GSEA tool was not explicitly provided in the text. \n(3) Yes, p-value correction was used in the functional enrichment analysis; specifically, a false discovery rate (FDR) correction was applied with alpha = 0.01. \n(4) Yes, over-representation analysis (hypergeometric test) was performed, and it was compared against all coding genes as the background gene list."
##
## $choices[[1]]$message$refusal
## NULL
##
## $choices[[1]]$message$annotations
## list()
##
##
## $choices[[1]]$logprobs
## NULL
##
## $choices[[1]]$finish_reason
## [1] "stop"
##
##
##
## $usage
## $usage$prompt_tokens
## [1] 12606
##
## $usage$completion_tokens
## [1] 113
##
## $usage$total_tokens
## [1] 12719
##
## $usage$prompt_tokens_details
## $usage$prompt_tokens_details$cached_tokens
## [1] 0
##
## $usage$prompt_tokens_details$audio_tokens
## [1] 0
##
##
## $usage$completion_tokens_details
## $usage$completion_tokens_details$reasoning_tokens
## [1] 0
##
## $usage$completion_tokens_details$audio_tokens
## [1] 0
##
## $usage$completion_tokens_details$accepted_prediction_tokens
## [1] 0
##
## $usage$completion_tokens_details$rejected_prediction_tokens
## [1] 0
##
##
##
## $service_tier
## [1] "default"
##
## $system_fingerprint
## [1] "fp_38343a2f8f"cat("LLM output:\n",response, "\n")## LLM output:
## (1) The bioinformatics tool used for pathway enrichment analysis was Gene Set Enrichment Analysis (GSEA).
## (2) The version number of the GSEA tool was not explicitly provided in the text.
## (3) Yes, p-value correction was used in the functional enrichment analysis; specifically, a false discovery rate (FDR) correction was applied with alpha = 0.01.
## (4) Yes, over-representation analysis (hypergeometric test) was performed, and it was compared against all coding genes as the background gene list.As you can see, ChatGPT has provided a response. On first glance it appears accurate, but we’ll need to confirm carefully.
We managed to get ChatGPT API calls working in R including the examination of journal articles in text format. This opens the door to other applications such as checklisting previous literature and helping journals to identify articles lacking methodological information.
sessionInfo()## R version 4.5.1 (2025-06-13)
## Platform: x86_64-pc-linux-gnu
## Running under: Ubuntu 24.04.3 LTS
##
## Matrix products: default
## BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
## LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.26.so; LAPACK version 3.12.0
##
## locale:
## [1] LC_CTYPE=en_AU.UTF-8 LC_NUMERIC=C
## [3] LC_TIME=en_AU.UTF-8 LC_COLLATE=en_AU.UTF-8
## [5] LC_MONETARY=en_AU.UTF-8 LC_MESSAGES=en_AU.UTF-8
## [7] LC_PAPER=en_AU.UTF-8 LC_NAME=C
## [9] LC_ADDRESS=C LC_TELEPHONE=C
## [11] LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C
##
## time zone: Australia/Melbourne
## tzcode source: system (glibc)
##
## attached base packages:
## [1] stats graphics grDevices utils datasets methods base
##
## other attached packages:
## [1] LLMR_0.5.0
##
## loaded via a namespace (and not attached):
## [1] vctrs_0.6.5 knitr_1.50 cli_3.6.5 xfun_0.52
## [5] rlang_1.1.6 purrr_1.1.0 generics_0.1.4 jsonlite_2.0.0
## [9] glue_1.8.0 htmltools_0.5.8.1 sass_0.4.10 rmarkdown_2.29
## [13] rappdirs_0.3.3 jquerylib_0.1.4 evaluate_1.0.4 tibble_3.3.0
## [17] fastmap_1.2.0 base64enc_0.1-3 yaml_2.3.10 lifecycle_1.0.4
## [21] httr2_1.2.1 memoise_2.0.1 compiler_4.5.1 dplyr_1.1.4
## [25] pkgconfig_2.0.3 tidyr_1.3.1 digest_0.6.37 R6_2.6.1
## [29] tidyselect_1.2.1 pillar_1.11.0 curl_6.4.0 magrittr_2.0.3
## [33] bslib_0.9.0 tools_4.5.1 mime_0.13 cachem_1.1.0