---
title: "Import model outputs"
author: "Benjamin Guiastrennec"
date: "`r format(Sys.time(), '%d %B, %Y')`"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Import model outputs}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
library(xpose)
options(width = 100)

xpdb <- xpdb_ex_pk

knitr::opts_chunk$set(fig.dpi = 96,
                      fig.align = 'center', 
                      fig.height = 4, 
                      fig.width = 4,
                      out.width = '75%',
                      comment = '',
                      message = FALSE)
```

### The `xpose_data` function
The function `xpose_data()` collects all model output files and table and organizes them into an R object commonly called `xpdb` for xpose data base. 

``` r
xpdb <- xpose_data(runno = '001')
```

### Imported files
#### NONMEM runs
To make full use of the functionality offered by `xpose` the following NONMEM output files should be available:

- `.lst/.out/.res`: used to collect information on the run (`template_titles`) as well as the output table names. Alternatively a model file (`.mod/.ctl`) can be used but some of the information in `template_titles` may not be available.
- `.ext`: used to collect final parameter estimates and residual standard error (RSE)
- `.phi`: used for the random effects and iOFV
- `.cov`: used for the covariance matrix
- `.cor`: used for the correlation matrix
- `.grd`: used for the estimation gradients
- `.shk`: used to compute random effect shrinkage `template_titles`
- output and simulation tables: for the actual data

When importing the files, xpose will return messages to the console and inform of any issue encountered during the import.

```r
xpdb <- xpose_data(runno = '001')

Looking for nonmem output tables
Reading: sdtab001, catab001, cotab001, patab001 [$prob no.1]

Looking for nonmem output files
Reading: run001.cor, run001.cov, run001.ext, run001.grd, run001.phi
```

These messages can be silenced with the option `quiet = TRUE`.

##### `FIRSTONLY` tables
xpose is compatible with the `$TABLE FIRSTONLY` option of NONMEM. The option `FIRSTONLY` only output the first record for each `ID` and hence can be used to decrease the size of output tables having no time-varying columns. During tables import xpose will merge `FIRSTONLY` tables with regular tables allowing seamless use of columns from `FIRSTONLY` in plots.

##### Compressed tables
xpose makes use of the [readr](https://readr.tidyverse.org) which offers the advantages of 1) rapidly import big tables and 2) automatically import compressed table (e.g. sdtab001.zip) to increase storage capacity.


#### Others NLME tools
Coming with future releases...


### Structure of the `xpdb` object
A typical `xpdb` object contains 8 levels namely:

- `code`: the parsed model code
- `summary`: contains key information regarding the model. All the information contained in the summary can be used as part of the `template_titles`.
- `data`: contains all output and simulation tables as well as the column indexing
- `files`: contains all output files
- `special`: contains post-processed datasets used by functions like `vpc()`
- `gg_theme`: an attached ggplot2 theme
- `xp_theme`: an attached xpose theme
- `options`: attached global options