Disseminating Prediction Methods

class: center, middle, inverse, title-slide

# Disseminating Prediction Methods
## Avoiding Computational Bottlenecks and Developing User-Friendly APIs
### Byron C Jaeger
### Wake Forest University School of Medicine
### 2022/08/08 (updated: 2022-08-07)

---

class: center, middle

<script>document.addEventListener('DOMContentLoaded',function(){new xeBanner(JSON.parse('{"left":"","right":"Byron C Jaeger","exclude":["inverse"],"position":"top"}'))})</script>
<script>document.addEventListener('DOMContentLoaded',function(){new xeBanner(JSON.parse('{"left":"byronjaeger.com/talk","exclude":["inverse"],"position":"bottom"}'))})</script>

# (Bio)statisticians create methods that can engage with contemporary data and make valid conclusions

---
class: center, middle

# Statistical software allows these methods to be shared with investigators

---
## From proprietary to open-source

When SAS and SPSS were prominent, methods were shared by incorporating them into proprietary software.

--

As R and Python have become standard languages for data science, it has become more common for authors to write their own software for methods.

---

## obliqueRSF

In 2019, I made the `obliqueRSF` R package

- oblique random survival forests

--

`obliqueRSF` had higher prediction accuracy versus:

- `randomForestSRC`, 
- `ranger`,
- `party`.

But was also hundreds of times slower.

---

<img src="index_files/figure-html/fig-time-one-1.png" style="display: block; margin: auto auto auto 0;" />

---
class: center, middle
background-image: url("img/meme_slow_R.jpg")
background-size: contain

---

<img src="index_files/figure-html/fig-time-two-1.png" style="display: block; margin: auto auto auto 0;" />

---

<img src="index_files/figure-html/fig-eval-cstat-1.png" style="display: block; margin: auto auto auto 0;" />

---

<img src="index_files/figure-html/fig-eval-ipa-1.png" style="display: block; margin: auto auto auto 0;" />

---
background-image: url("img/aorsf-arxiv-paper.png")
background-size: 85%

## aorsf paper (https://arxiv.org/abs/2208.01129)

---
background-image: url("img/aorsf-arxiv-qr.png")
background-size: 65%

---
class: center, middle, inverse

# Four guidelines for faster code

---
class: center, middle

# 1. Always be benchmarking

---
class: center, middle

# Why "always"?

---
## How to benchmark?

__Step 1. Define a task:__ Count events per group (status of 1 `\(\Rightarrow\)` event, 3 groups)

```r
# status = c(1, 0, 1, ...)
status = sample(x = c(0L, 1L), size = 1e5, replace = TRUE)

# group = c(2, 0, 1, ...)
group = sample(x = c(0L, 1L, 2L), size = 1e5, replace = TRUE)
```

--

__Step 2. Define the reference competitors__

```r
# competitors: table() and tapply()
table(status, group)[2, ] 
```

```
##     0     1     2 
## 16836 16708 16831
```

```r
tapply(status, group, FUN = sum)
```

```
##     0     1     2 
## 16836 16708 16831
```

---

## How to benchmark?

__Step 3 (optional).__ Enter your own competitor(s)!

<img src="ink/vecs_1.png" width="2063" />

---

## How to benchmark?

__Step 3 (optional).__ Enter your own competitor(s)!

<img src="ink/vecs_2.png" width="2063" />

---

## How to benchmark?

__Step 3 (optional).__ Enter your own competitor(s)!

<img src="ink/vecs_3.png" width="2063" />

---

## How to benchmark?

__Step 3 (optional).__ Enter your own competitor(s)!

<img src="ink/vecs_4.png" width="2063" />

---

## How to benchmark?

__Step 3 (optional).__ Enter your own competitor(s)!

```cpp
#include <Rcpp.h>
using namespace Rcpp;

// [[Rcpp::export]]
NumericVector rcpp_count_dbl(NumericVector status,
                             NumericVector group,
                             int n_groups) {
 
 NumericVector out(n_groups);
 
 for( int i = 0; i < n_groups; i++ ){
  for( int j = 0; j < group.length(); j++ ){
   if(group[j] == i) out[i] += status[j];
  }
 }
 
 return(out);
}

```

---

## How to benchmark?

__Step 4. Off to the races:__

```r
library(microbenchmark)
microbenchmark(table = table(status, group)[2, ],
               tapply = tapply(status, group, FUN = sum),
               rcpp_count_dbl = rcpp_count_dbl(status, group, 3))
```

--

<div id="wzalsepbtm" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>html {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif;
}

#wzalsepbtm .gt_table {
  display: table;
  border-collapse: collapse;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#wzalsepbtm .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#wzalsepbtm .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#wzalsepbtm .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 0;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#wzalsepbtm .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#wzalsepbtm .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#wzalsepbtm .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#wzalsepbtm .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#wzalsepbtm .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#wzalsepbtm .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#wzalsepbtm .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 5px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#wzalsepbtm .gt_group_heading {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
}

#wzalsepbtm .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#wzalsepbtm .gt_from_md > :first-child {
  margin-top: 0;
}

#wzalsepbtm .gt_from_md > :last-child {
  margin-bottom: 0;
}

#wzalsepbtm .gt_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#wzalsepbtm .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
}

#wzalsepbtm .gt_stub_row_group {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
  vertical-align: top;
}

#wzalsepbtm .gt_row_group_first td {
  border-top-width: 2px;
}

#wzalsepbtm .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#wzalsepbtm .gt_first_summary_row {
  border-top-style: solid;
  border-top-color: #D3D3D3;
}

#wzalsepbtm .gt_first_summary_row.thick {
  border-top-width: 2px;
}

#wzalsepbtm .gt_last_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#wzalsepbtm .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#wzalsepbtm .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#wzalsepbtm .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#wzalsepbtm .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#wzalsepbtm .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#wzalsepbtm .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding-left: 4px;
  padding-right: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#wzalsepbtm .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#wzalsepbtm .gt_sourcenote {
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#wzalsepbtm .gt_left {
  text-align: left;
}

#wzalsepbtm .gt_center {
  text-align: center;
}

#wzalsepbtm .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#wzalsepbtm .gt_font_normal {
  font-weight: normal;
}

#wzalsepbtm .gt_font_bold {
  font-weight: bold;
}

#wzalsepbtm .gt_font_italic {
  font-style: italic;
}

#wzalsepbtm .gt_super {
  font-size: 65%;
}

#wzalsepbtm .gt_two_val_uncert {
  display: inline-block;
  line-height: 1em;
  text-align: right;
  font-size: 60%;
  vertical-align: -0.25em;
  margin-left: 0.1em;
}

#wzalsepbtm .gt_footnote_marks {
  font-style: italic;
  font-weight: normal;
  font-size: 75%;
  vertical-align: 0.4em;
}

#wzalsepbtm .gt_asterisk {
  font-size: 100%;
  vertical-align: 0;
}

#wzalsepbtm .gt_slash_mark {
  font-size: 0.7em;
  line-height: 0.7em;
  vertical-align: 0.15em;
}

#wzalsepbtm .gt_fraction_numerator {
  font-size: 0.6em;
  line-height: 0.6em;
  vertical-align: 0.45em;
}

#wzalsepbtm .gt_fraction_denominator {
  font-size: 0.6em;
  line-height: 0.6em;
  vertical-align: -0.05em;
}
</style>
<table class="gt_table">
  <thead class="gt_header">
    <tr>
      <th colspan="7" class="gt_heading gt_title gt_font_normal" style>Benchmark demonstration: counting events in groups</th>
    </tr>
    <tr>
      <th colspan="7" class="gt_heading gt_subtitle gt_font_normal gt_bottom_border" style>table(), tapply(), and rcpp_count_dbl()</th>
    </tr>
  </thead>
  <thead class="gt_col_headings">
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="2" colspan="1">Function</th>
      <th class="gt_center gt_columns_top_border gt_column_spanner_outer" rowspan="1" colspan="6">
        <span class="gt_column_spanner">Time, milliseconds</span>
      </th>
    </tr>
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Minimum</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">25th %</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Mean</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Median</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">75th %</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Maximum</th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr><td class="gt_row gt_right gt_stub">table</td>
<td class="gt_row gt_center">5.5</td>
<td class="gt_row gt_center">6.0</td>
<td class="gt_row gt_center">7.8</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">6.1</td>
<td class="gt_row gt_center">6.3</td>
<td class="gt_row gt_center">73</td></tr>
    <tr><td class="gt_row gt_right gt_stub">tapply</td>
<td class="gt_row gt_center">2.1</td>
<td class="gt_row gt_center">2.5</td>
<td class="gt_row gt_center">3.2</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">2.5</td>
<td class="gt_row gt_center">2.6</td>
<td class="gt_row gt_center">81</td></tr>
    <tr><td class="gt_row gt_right gt_stub">rcpp_count_dbl</td>
<td class="gt_row gt_center">1.6</td>
<td class="gt_row gt_center">1.8</td>
<td class="gt_row gt_center">2.0</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">1.8</td>
<td class="gt_row gt_center">1.8</td>
<td class="gt_row gt_center">18</td></tr>
  </tbody>
  
  
</table>
</div>

---
class: center, middle

# 2. Trace your data

---
## Trace your data

__Definition__ be notified when data are copied or cast to a different type

--

(Copying and casting require additional memory, slowing down your code.)

---
## How to trace?

Put the object you want to trace into `tracemem()`:

```r
tracemem(status)
tracemem(group)
```

R will now notify you if `status` or `group` are copied or cast to a different type.

---
## Here is a problem

`rcpp_count_dbl` casts both `status` and `group` from integer to double

```r
rcpp_count_dbl(status, group, n_groups = 3)
```

```
## tracemem[0x00007ff4fd830010 -> 0x00007ff4fd660010]: .Call rcpp_count_dbl eval eval eval_with_user_handlers withVisible withCallingHandlers handle timing_fn evaluate_call <Anonymous> evaluate in_dir in_input_dir eng_r block_exec call_block process_group.block process_group withCallingHandlers process_file <Anonymous> <Anonymous> 
## tracemem[0x00007ff4fd910010 -> 0x00007ff4fd3a0010]: .Call rcpp_count_dbl eval eval eval_with_user_handlers withVisible withCallingHandlers handle timing_fn evaluate_call <Anonymous> evaluate in_dir in_input_dir eng_r block_exec call_block process_group.block process_group withCallingHandlers process_file <Anonymous> <Anonymous>
```

```
## [1] 16836 16708 16831
```

---
## Here is a problem

Our first `Rcpp` function expected numeric vectors. We gave it integers!

```cpp
#include <Rcpp.h>
using namespace Rcpp;

// [[Rcpp::export]]
*NumericVector rcpp_count_dbl(NumericVector status,
*                            NumericVector group,
                             int n_groups) {
 
*NumericVector out(n_groups);
 
 for( int i = 0; i < n_groups; i++ ){
  for( int j = 0; j < group.length(); j++ ){
   if(group[j] == i) out[i] += status[j];
  }
 }
 
 return(out);
}

```

---
## Let's fix that

use __integerVector__ instead of __numericVector__

```cpp
#include <Rcpp.h>
using namespace Rcpp;

// [[Rcpp::export]]
*IntegerVector rcpp_count_int(IntegerVector status,
*                            IntegerVector group,
                             int n_groups) {
 
*IntegerVector out(n_groups);
 
 for( int i = 0; i < n_groups; i++ ){
  for( int j = 0; j < group.length(); j++ ){
   if(group[j] == i) out[i] += status[j];
  }
 }
 
 return(out);
}

```

---
## Better

`tracemem()` has been pacified!

```r
rcpp_count_int(status, group, n_groups = 3)
```

```
## [1] 16836 16708 16831
```

---
class: center, middle

## Faster

<div id="sxqpsthmcg" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>html {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif;
}

#sxqpsthmcg .gt_table {
  display: table;
  border-collapse: collapse;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#sxqpsthmcg .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#sxqpsthmcg .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#sxqpsthmcg .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 0;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#sxqpsthmcg .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#sxqpsthmcg .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#sxqpsthmcg .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#sxqpsthmcg .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#sxqpsthmcg .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#sxqpsthmcg .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#sxqpsthmcg .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 5px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#sxqpsthmcg .gt_group_heading {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
}

#sxqpsthmcg .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#sxqpsthmcg .gt_from_md > :first-child {
  margin-top: 0;
}

#sxqpsthmcg .gt_from_md > :last-child {
  margin-bottom: 0;
}

#sxqpsthmcg .gt_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#sxqpsthmcg .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
}

#sxqpsthmcg .gt_stub_row_group {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
  vertical-align: top;
}

#sxqpsthmcg .gt_row_group_first td {
  border-top-width: 2px;
}

#sxqpsthmcg .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#sxqpsthmcg .gt_first_summary_row {
  border-top-style: solid;
  border-top-color: #D3D3D3;
}

#sxqpsthmcg .gt_first_summary_row.thick {
  border-top-width: 2px;
}

#sxqpsthmcg .gt_last_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#sxqpsthmcg .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#sxqpsthmcg .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#sxqpsthmcg .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#sxqpsthmcg .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#sxqpsthmcg .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#sxqpsthmcg .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding-left: 4px;
  padding-right: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#sxqpsthmcg .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#sxqpsthmcg .gt_sourcenote {
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#sxqpsthmcg .gt_left {
  text-align: left;
}

#sxqpsthmcg .gt_center {
  text-align: center;
}

#sxqpsthmcg .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#sxqpsthmcg .gt_font_normal {
  font-weight: normal;
}

#sxqpsthmcg .gt_font_bold {
  font-weight: bold;
}

#sxqpsthmcg .gt_font_italic {
  font-style: italic;
}

#sxqpsthmcg .gt_super {
  font-size: 65%;
}

#sxqpsthmcg .gt_two_val_uncert {
  display: inline-block;
  line-height: 1em;
  text-align: right;
  font-size: 60%;
  vertical-align: -0.25em;
  margin-left: 0.1em;
}

#sxqpsthmcg .gt_footnote_marks {
  font-style: italic;
  font-weight: normal;
  font-size: 75%;
  vertical-align: 0.4em;
}

#sxqpsthmcg .gt_asterisk {
  font-size: 100%;
  vertical-align: 0;
}

#sxqpsthmcg .gt_slash_mark {
  font-size: 0.7em;
  line-height: 0.7em;
  vertical-align: 0.15em;
}

#sxqpsthmcg .gt_fraction_numerator {
  font-size: 0.6em;
  line-height: 0.6em;
  vertical-align: 0.45em;
}

#sxqpsthmcg .gt_fraction_denominator {
  font-size: 0.6em;
  line-height: 0.6em;
  vertical-align: -0.05em;
}
</style>
<table class="gt_table">
  <thead class="gt_header">
    <tr>
      <th colspan="7" class="gt_heading gt_title gt_font_normal" style>Benchmark demonstration: counting events in groups</th>
    </tr>
    <tr>
      <th colspan="7" class="gt_heading gt_subtitle gt_font_normal gt_bottom_border" style>table(), tapply(), and Rcpp functions</th>
    </tr>
  </thead>
  <thead class="gt_col_headings">
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="2" colspan="1">Function</th>
      <th class="gt_center gt_columns_top_border gt_column_spanner_outer" rowspan="1" colspan="6">
        <span class="gt_column_spanner">Time, milliseconds</span>
      </th>
    </tr>
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Minimum</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">25th %</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Mean</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Median</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">75th %</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Maximum</th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr><td class="gt_row gt_right gt_stub">table</td>
<td class="gt_row gt_center">5.5</td>
<td class="gt_row gt_center">6.0</td>
<td class="gt_row gt_center">7.8</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">6.1</td>
<td class="gt_row gt_center">6.3</td>
<td class="gt_row gt_center">73</td></tr>
    <tr><td class="gt_row gt_right gt_stub">tapply</td>
<td class="gt_row gt_center">2.1</td>
<td class="gt_row gt_center">2.5</td>
<td class="gt_row gt_center">3.2</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">2.5</td>
<td class="gt_row gt_center">2.6</td>
<td class="gt_row gt_center">81</td></tr>
    <tr><td class="gt_row gt_right gt_stub">rcpp_count_dbl</td>
<td class="gt_row gt_center">1.6</td>
<td class="gt_row gt_center">1.8</td>
<td class="gt_row gt_center">2.0</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">1.8</td>
<td class="gt_row gt_center">1.8</td>
<td class="gt_row gt_center">18</td></tr>
    <tr><td class="gt_row gt_right gt_stub">rcpp_count_int</td>
<td class="gt_row gt_center">1.4</td>
<td class="gt_row gt_center">1.4</td>
<td class="gt_row gt_center">1.4</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">1.4</td>
<td class="gt_row gt_center">1.4</td>
<td class="gt_row gt_center">2.4</td></tr>
  </tbody>
  
  
</table>
</div>

---
class: center, middle

# 3. Count your operations

(Number of operations `\(\approx\)` speed of your code)

---
## How to count

You don't have to be exact - think big picture

---

## How to count

In our C++ function, we use `\(n\)` operations for each unique value in `group`,

<img src="ink/vecs_1.png" width="2063" />

---
## How to count

In our C++ function, we use `\(n\)` operations for each unique value in `group`,

<img src="ink/vecs_2.png" width="2063" />

---
## How to count

In our C++ function, we use `\(n\)` operations for each unique value in `group`,

<img src="ink/vecs_3.png" width="2063" />

---
## How to count

In our C++ function, we use `\(n\)` operations for each unique value in `group`,

<img src="ink/vecs_4.png" width="2063" />

---
## How to count

As `\(n, g \rightarrow \infty\)`, we use `\(\mathcal{O}(n \cdot g)\)` operations, where `\(g\)` = number of groups

--

Can we reduce the operation cost?

---
## 1 loop instead of 2

<img src="ink/vecs_5.png" width="2263" />

---
## 1 loop instead of 2

<img src="ink/vecs_6.png" width="2263" />

---
## 1 loop instead of 2

<img src="ink/vecs_7.png" width="2263" />

---
## 1 loop instead of 2

<img src="ink/vecs_8.png" width="2263" />

---
## 1 loop instead of 2

<img src="ink/vecs_9.png" width="2263" />

---
## 1 loop instead of 2

code adapted from `rcpp_count_int`

```cpp
#include <Rcpp.h>
using namespace Rcpp;

// [[Rcpp::export]]
IntegerVector rcpp_count_1loop_int(IntegerVector status,
                                   IntegerVector group,
                                   int n_groups) {
 
 IntegerVector out(n_groups);
 IntegerVector::iterator i;
 int j = 0;
 
 for(i = group.begin() ; i != group.end(); ++i, ++j){
  out[*i] += status[j];
 }
 
 return(out);
}

```

---
class: center, middle

## Much faster!

<div id="gdxdcnlrss" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>html {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif;
}

#gdxdcnlrss .gt_table {
  display: table;
  border-collapse: collapse;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#gdxdcnlrss .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#gdxdcnlrss .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#gdxdcnlrss .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 0;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#gdxdcnlrss .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#gdxdcnlrss .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#gdxdcnlrss .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#gdxdcnlrss .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#gdxdcnlrss .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#gdxdcnlrss .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#gdxdcnlrss .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 5px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#gdxdcnlrss .gt_group_heading {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
}

#gdxdcnlrss .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#gdxdcnlrss .gt_from_md > :first-child {
  margin-top: 0;
}

#gdxdcnlrss .gt_from_md > :last-child {
  margin-bottom: 0;
}

#gdxdcnlrss .gt_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#gdxdcnlrss .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
}

#gdxdcnlrss .gt_stub_row_group {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
  vertical-align: top;
}

#gdxdcnlrss .gt_row_group_first td {
  border-top-width: 2px;
}

#gdxdcnlrss .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#gdxdcnlrss .gt_first_summary_row {
  border-top-style: solid;
  border-top-color: #D3D3D3;
}

#gdxdcnlrss .gt_first_summary_row.thick {
  border-top-width: 2px;
}

#gdxdcnlrss .gt_last_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#gdxdcnlrss .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#gdxdcnlrss .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#gdxdcnlrss .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#gdxdcnlrss .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#gdxdcnlrss .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#gdxdcnlrss .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding-left: 4px;
  padding-right: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#gdxdcnlrss .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#gdxdcnlrss .gt_sourcenote {
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#gdxdcnlrss .gt_left {
  text-align: left;
}

#gdxdcnlrss .gt_center {
  text-align: center;
}

#gdxdcnlrss .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#gdxdcnlrss .gt_font_normal {
  font-weight: normal;
}

#gdxdcnlrss .gt_font_bold {
  font-weight: bold;
}

#gdxdcnlrss .gt_font_italic {
  font-style: italic;
}

#gdxdcnlrss .gt_super {
  font-size: 65%;
}

#gdxdcnlrss .gt_two_val_uncert {
  display: inline-block;
  line-height: 1em;
  text-align: right;
  font-size: 60%;
  vertical-align: -0.25em;
  margin-left: 0.1em;
}

#gdxdcnlrss .gt_footnote_marks {
  font-style: italic;
  font-weight: normal;
  font-size: 75%;
  vertical-align: 0.4em;
}

#gdxdcnlrss .gt_asterisk {
  font-size: 100%;
  vertical-align: 0;
}

#gdxdcnlrss .gt_slash_mark {
  font-size: 0.7em;
  line-height: 0.7em;
  vertical-align: 0.15em;
}

#gdxdcnlrss .gt_fraction_numerator {
  font-size: 0.6em;
  line-height: 0.6em;
  vertical-align: 0.45em;
}

#gdxdcnlrss .gt_fraction_denominator {
  font-size: 0.6em;
  line-height: 0.6em;
  vertical-align: -0.05em;
}
</style>
<table class="gt_table">
  <thead class="gt_header">
    <tr>
      <th colspan="7" class="gt_heading gt_title gt_font_normal" style>Benchmark demonstration: counting events in groups</th>
    </tr>
    <tr>
      <th colspan="7" class="gt_heading gt_subtitle gt_font_normal gt_bottom_border" style>table(), tapply(), and Rcpp functions</th>
    </tr>
  </thead>
  <thead class="gt_col_headings">
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="2" colspan="1">Function</th>
      <th class="gt_center gt_columns_top_border gt_column_spanner_outer" rowspan="1" colspan="6">
        <span class="gt_column_spanner">Time, milliseconds</span>
      </th>
    </tr>
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Minimum</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">25th %</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Mean</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Median</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">75th %</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Maximum</th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr><td class="gt_row gt_right gt_stub">table</td>
<td class="gt_row gt_center">5.5</td>
<td class="gt_row gt_center">6.0</td>
<td class="gt_row gt_center">7.8</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">6.1</td>
<td class="gt_row gt_center">6.3</td>
<td class="gt_row gt_center">73</td></tr>
    <tr><td class="gt_row gt_right gt_stub">tapply</td>
<td class="gt_row gt_center">2.1</td>
<td class="gt_row gt_center">2.5</td>
<td class="gt_row gt_center">3.2</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">2.5</td>
<td class="gt_row gt_center">2.6</td>
<td class="gt_row gt_center">81</td></tr>
    <tr><td class="gt_row gt_right gt_stub">rcpp_count_dbl</td>
<td class="gt_row gt_center">1.6</td>
<td class="gt_row gt_center">1.8</td>
<td class="gt_row gt_center">2.0</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">1.8</td>
<td class="gt_row gt_center">1.8</td>
<td class="gt_row gt_center">18</td></tr>
    <tr><td class="gt_row gt_right gt_stub">rcpp_count_int</td>
<td class="gt_row gt_center">1.4</td>
<td class="gt_row gt_center">1.4</td>
<td class="gt_row gt_center">1.4</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">1.4</td>
<td class="gt_row gt_center">1.4</td>
<td class="gt_row gt_center">2.4</td></tr>
    <tr><td class="gt_row gt_right gt_stub">rcpp_count_1loop_int</td>
<td class="gt_row gt_center">0.18</td>
<td class="gt_row gt_center">0.19</td>
<td class="gt_row gt_center">0.19</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">0.19</td>
<td class="gt_row gt_center">0.19</td>
<td class="gt_row gt_center">1.1</td></tr>
  </tbody>
  
  
</table>
</div>

---
class: center, middle

# 4. Ride the Armadillo

---
background-image: url("img/meme_aorsf_car.png")
background-size: 65%

---
background-image: url("img/arma.png")
background-size: 90%

## Armadillo (http://arma.sourceforge.net/)

---
## Armadillo

```cpp
#include <RcppArmadillo.h>
#include <RcppArmadilloExtensions/sample.h>
// [[Rcpp::depends(RcppArmadillo)]]
using namespace Rcpp;
using namespace arma;

// [[Rcpp::export]]
arma::ivec arma_count_1loop_int(arma::ivec& status,
                                arma::ivec& group,
                                arma::uword n_groups) {
 
 ivec out(n_groups);
 ivec::iterator i;
 uword j = 0;
 
 for(i = group.begin() ; i != group.end(); ++i, ++j){
  out[*i] += status[j];
 }
 
 return(out);
 
}

```

---
class: center, middle

## Better! (almost perfect)

<div id="yufsqprell" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>html {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif;
}

#yufsqprell .gt_table {
  display: table;
  border-collapse: collapse;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#yufsqprell .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#yufsqprell .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#yufsqprell .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 0;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#yufsqprell .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#yufsqprell .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#yufsqprell .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#yufsqprell .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#yufsqprell .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#yufsqprell .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#yufsqprell .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 5px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#yufsqprell .gt_group_heading {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
}

#yufsqprell .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#yufsqprell .gt_from_md > :first-child {
  margin-top: 0;
}

#yufsqprell .gt_from_md > :last-child {
  margin-bottom: 0;
}

#yufsqprell .gt_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#yufsqprell .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
}

#yufsqprell .gt_stub_row_group {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
  vertical-align: top;
}

#yufsqprell .gt_row_group_first td {
  border-top-width: 2px;
}

#yufsqprell .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#yufsqprell .gt_first_summary_row {
  border-top-style: solid;
  border-top-color: #D3D3D3;
}

#yufsqprell .gt_first_summary_row.thick {
  border-top-width: 2px;
}

#yufsqprell .gt_last_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#yufsqprell .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#yufsqprell .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#yufsqprell .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#yufsqprell .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#yufsqprell .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#yufsqprell .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding-left: 4px;
  padding-right: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#yufsqprell .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#yufsqprell .gt_sourcenote {
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#yufsqprell .gt_left {
  text-align: left;
}

#yufsqprell .gt_center {
  text-align: center;
}

#yufsqprell .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#yufsqprell .gt_font_normal {
  font-weight: normal;
}

#yufsqprell .gt_font_bold {
  font-weight: bold;
}

#yufsqprell .gt_font_italic {
  font-style: italic;
}

#yufsqprell .gt_super {
  font-size: 65%;
}

#yufsqprell .gt_two_val_uncert {
  display: inline-block;
  line-height: 1em;
  text-align: right;
  font-size: 60%;
  vertical-align: -0.25em;
  margin-left: 0.1em;
}

#yufsqprell .gt_footnote_marks {
  font-style: italic;
  font-weight: normal;
  font-size: 75%;
  vertical-align: 0.4em;
}

#yufsqprell .gt_asterisk {
  font-size: 100%;
  vertical-align: 0;
}

#yufsqprell .gt_slash_mark {
  font-size: 0.7em;
  line-height: 0.7em;
  vertical-align: 0.15em;
}

#yufsqprell .gt_fraction_numerator {
  font-size: 0.6em;
  line-height: 0.6em;
  vertical-align: 0.45em;
}

#yufsqprell .gt_fraction_denominator {
  font-size: 0.6em;
  line-height: 0.6em;
  vertical-align: -0.05em;
}
</style>
<table class="gt_table">
  <thead class="gt_header">
    <tr>
      <th colspan="7" class="gt_heading gt_title gt_font_normal" style>Benchmark demonstration: counting events in groups</th>
    </tr>
    <tr>
      <th colspan="7" class="gt_heading gt_subtitle gt_font_normal gt_bottom_border" style>table(), tapply(), and Rcpp functions</th>
    </tr>
  </thead>
  <thead class="gt_col_headings">
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="2" colspan="1">Function</th>
      <th class="gt_center gt_columns_top_border gt_column_spanner_outer" rowspan="1" colspan="6">
        <span class="gt_column_spanner">Time, milliseconds</span>
      </th>
    </tr>
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Minimum</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">25th %</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Mean</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Median</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">75th %</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1">Maximum</th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr><td class="gt_row gt_right gt_stub">table</td>
<td class="gt_row gt_center">5.5</td>
<td class="gt_row gt_center">6.0</td>
<td class="gt_row gt_center">7.8</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">6.1</td>
<td class="gt_row gt_center">6.3</td>
<td class="gt_row gt_center">73</td></tr>
    <tr><td class="gt_row gt_right gt_stub">tapply</td>
<td class="gt_row gt_center">2.1</td>
<td class="gt_row gt_center">2.5</td>
<td class="gt_row gt_center">3.2</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">2.5</td>
<td class="gt_row gt_center">2.6</td>
<td class="gt_row gt_center">81</td></tr>
    <tr><td class="gt_row gt_right gt_stub">rcpp_count_dbl</td>
<td class="gt_row gt_center">1.6</td>
<td class="gt_row gt_center">1.8</td>
<td class="gt_row gt_center">2.0</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">1.8</td>
<td class="gt_row gt_center">1.8</td>
<td class="gt_row gt_center">18</td></tr>
    <tr><td class="gt_row gt_right gt_stub">rcpp_count_int</td>
<td class="gt_row gt_center">1.4</td>
<td class="gt_row gt_center">1.4</td>
<td class="gt_row gt_center">1.4</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">1.4</td>
<td class="gt_row gt_center">1.4</td>
<td class="gt_row gt_center">2.4</td></tr>
    <tr><td class="gt_row gt_right gt_stub">rcpp_count_1loop_int</td>
<td class="gt_row gt_center">0.18</td>
<td class="gt_row gt_center">0.19</td>
<td class="gt_row gt_center">0.19</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">0.19</td>
<td class="gt_row gt_center">0.19</td>
<td class="gt_row gt_center">1.1</td></tr>
    <tr><td class="gt_row gt_right gt_stub">arma_count_1loop_int</td>
<td class="gt_row gt_center">0.07</td>
<td class="gt_row gt_center">0.07</td>
<td class="gt_row gt_center">0.08</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">0.07</td>
<td class="gt_row gt_center">0.08</td>
<td class="gt_row gt_center">1.0</td></tr>
    <tr><td class="gt_row gt_right gt_stub">sum</td>
<td class="gt_row gt_center">0.05</td>
<td class="gt_row gt_center">0.05</td>
<td class="gt_row gt_center">0.05</td>
<td class="gt_row gt_center" style="background-color: #F9E3D6; font-style: italic;">0.05</td>
<td class="gt_row gt_center">0.05</td>
<td class="gt_row gt_center">0.13</td></tr>
  </tbody>
  
  
</table>
</div>

---
class: inverse, center, middle

# Creating a Friendly API

---

## Names

__be consistent__: use _one_ convention throughout your package

- `snake_case`
- `camelCase`
- `SCREAMING_SNAKE`
- `ConFuse.EVERY_one` (don't do it)

Name functions with a __Noun__, then a __verb__, then __details__ (helps auto-completion & creates function families)

- `orsf_vi_negate()`, `orsf_vi_anova()`, `orsf_vi_permute()`
- `orsf_pd_ice()`, `orsf_pd_summary()`
- `orsf_control_cph()`, `orsf_control_net()`

__Exception:__ the modeling function is just the model name:

- `lm()`
- `glm()`
- `orsf()`

---

## Check arguments

Vet inputs and write error messages that spark joy

```r
library(aorsf)

*pbc_orsf$date_var <- lubridate::today()
*pbc_orsf$char_var <- "I'm a character"

fit <- orsf(
 data = pbc_orsf, 
 formula = Surv(time, status) ~ date_var + char_var
) 
```

```
## Error: some variables have unsupported type:
##  <date_var> has type <Date>
##  <char_var> has type <character>
## supported types are numeric, integer, units, factor, and ordered
```

---

## Generic methods

Include a generic `print` function (so your users won't hate you)

```r
print(fit)
```

```
## ---------- Oblique random survival forest
## 
##           N observations: 276
##                 N events: 111
##                  N trees: 500
##       N predictors total: 17
##    N predictors per node: 5
##  Average leaves per tree: 24
## Min observations in leaf: 5
##       Min events in leaf: 1
##           OOB stat value: 0.84
##            OOB stat type: Harrell's C-statistic
## 
## -----------------------------------------
```

---

## Generic methods

Include a generic `print` function (so your users won't hate you)

```r
print(unclass(fit))
```

```
## $forest
## $forest[[1]]
## $forest[[1]]$leaf_nodes
##       [,1]      [,2]
##  [1,]  853 0.8333333
##  [2,]  859 0.5000000
##  [3,]  930 0.0000000
##  [4,]   41 0.8750000
##  [5,]  334 0.6250000
##  [6,]  348 0.5000000
##  [7,]  400 0.3750000
##  [8,]  549 0.2500000
##  [9,] 1191 0.0000000
## [10,]   51 0.8571429
## [11,]  131 0.5714286
## [12,]  179 0.4285714
## [13,]  223 0.2857143
## [14,]  264 0.0000000
## [15,] 1536 0.9861111
## [16,] 2055 0.9488994
## [17,] 2090 0.9302935
## [18,] 2598 0.8970687
## [19,] 3839 0.0000000
## [20,]  999 0.7500000
## [21,] 1690 0.2500000
## [22,] 1925 0.0000000
## [23,] 2466 0.8750000
## [24,] 3853 0.6562500
## [25,] 1212 0.6250000
## [26,] 1847 0.4687500
## [27,] 2769 0.3125000
## [28,] 3170 0.1562500
## [29,]  191 0.8000000
## [30,]  388 0.4000000
## [31,] 1434 0.2000000
## [32,]  799 0.8000000
## [33,]  974 0.2000000
## [34,]  198 0.6000000
## [35,]  515 0.2000000
## [36,] 1012 0.8750000
## [37,] 2540 0.3750000
## [38,] 3282 0.1250000
## [39,] 3574 0.0000000
## [40,] 1152 0.8888889
## [41,] 1297 0.5555556
## [42,] 1827 0.1851852
## [43,] 2386 0.0000000
## [44,]  790 0.6666667
## [45,]  980 0.5000000
## [46,] 1356 0.0000000
## [47,] 3244 0.0000000
## [48,]  186 0.8333333
## [49,]  611 0.6666667
## [50,]  850 0.3333333
## [51,] 1077 0.1666667
## [52,] 1576 0.0000000
## [53,]  304 0.4000000
## [54,]  321 0.2000000
## [55,]  750 0.0000000
## [56,]  597 0.8000000
## [57,]  673 0.6000000
## [58,] 1080 0.4000000
## [59,] 4191 0.6666667
## [60,] 3086 0.2500000
## [61,] 4079 0.0000000
## [62,] 1360 0.5000000
## [63,] 2689 0.0000000
## [64,]  786 0.8000000
## [65,] 1165 0.6000000
## [66,] 1235 0.4000000
## [67,] 1690 0.0000000
## [68,] 2256 0.0000000
## [69,] 1413 0.6000000
## [70,] 1427 0.0000000
## [71,] 3445 0.5000000
## [72,] 3584 0.2500000
## [73,] 3762 0.1250000
## 
## $forest[[1]]$leaf_node_index
##       [,1] [,2] [,3]
##  [1,]   12    0    2
##  [2,]   13    3    8
##  [3,]   14    9   13
##  [4,]    7   14   17
##  [5,]   15   18   18
##  [6,]   16   19   21
##  [7,]   17   22   23
##  [8,]   18   24   27
##  [9,]   20   28   30
## [10,]   22   31   32
## [11,]   28   33   34
## [12,]   31   35   38
## [13,]   32   39   42
## [14,]   34   43   45
## [15,]   37   46   46
## [16,]   39   47   51
## [17,]   40   52   54
## [18,]   42   55   57
## [19,]   43   58   58
## [20,]   45   59   60
## [21,]   46   61   62
## [22,]   38   63   66
## [23,]   47   67   67
## [24,]   48   68   69
## [25,]   44   70   72
## 
## $forest[[1]]$betas
##             [,1]         [,2]          [,3]        [,4]         [,5]
## [1,] 0.262985825  0.725474028 -0.0014931813  0.94992351  0.006112226
## [2,] 0.158193898  0.004359448 -0.0005942595  0.53572175 -0.368080944
## [3,] 0.008543901 13.969268915  1.7872001887  0.05005997 -0.001765337
## [4,] 0.029681631 -0.322230731  0.9396864685 -0.50915286  0.750941521
## [5,] 0.045705758  0.003259517  0.0110829512  0.00000000  0.252493500
##               [,6]          [,7] [,8]        [,9]       [,10]        [,11]
## [1,] -5.650676e-05 -2.511740e+00    0 -3.49412714  0.39729552 -0.004885386
## [2,]  8.456075e-01 -3.423573e-03    0 -0.07339156  0.05804965 -0.002203610
## [3,] -3.723265e-02  6.201833e-05    0  0.00000000 -0.01866679  4.350385748
## [4,]  7.066679e-03 -3.358876e-01    0  0.00000000  0.00000000 -0.002155869
## [5,]  1.536013e-01  2.943154e-03    0  0.00000000  0.00000000 -0.002239028
##             [,12] [,13] [,14] [,15] [,16] [,17] [,18] [,19]        [,20] [,21]
## [1,] -0.514142372     0     0     0     0     0     0     0 -0.462549496     0
## [2,]  0.790376936     0     0     0     0     0     0     0  0.497797737     0
## [3,] -0.401328688     0     0     0     0     0     0     0  0.000852033     0
## [4,]  0.979522952     0     0     0     0     0     0     0 -0.000212850     0
## [5,] -0.003196098     0     0     0     0     0     0     0  0.647760398     0
##              [,22] [,23]        [,24]         [,25]        [,26]         [,27]
## [1,] -0.0318685407     0  0.005395293 -0.0076221656  0.012066060 -0.0044115983
## [2,]  0.0063563407     0 -0.463340846  0.0003262873  0.322261169 -0.0588258725
## [3,] -0.0006930693     0  0.006615146  0.0009290306 -0.021175594 -0.4307647310
## [4,]  0.4374105755     0  0.046328187  1.4522319550  0.002408217 -0.0002553339
## [5,]  0.7559888620     0  0.001594554 -0.6085498876 -2.937782840 -0.0171102718
##              [,28] [,29]        [,30]         [,31] [,32] [,33]      [,34]
## [1,]  0.0002100783     0  0.205013123 -0.0176893511     0     0  0.2666386
## [2,] -0.3311371928     0 -0.001299012  0.0001557652     0     0  1.4257512
## [3,] -0.0182356208     0  0.236369518  0.9422256826     0     0 -3.0912894
## [4,] -0.1720941062     0  0.000000000  0.0000000000     0     0  0.0000000
## [5,]  0.0076723581     0  0.000000000  0.0000000000     0     0  0.0000000
##      [,35]        [,36]       [,37] [,38] [,39] [,40] [,41]        [,42] [,43]
## [1,]     0 -0.004318706  3.48094343     0     0     0     0 -1.594649988     0
## [2,]     0 -0.005317667 -0.04349577     0     0     0     0  0.002894705     0
## [3,]     0  0.000000000  0.00000000     0     0     0     0  0.000000000     0
## [4,]     0  0.000000000  0.00000000     0     0     0     0  0.000000000     0
## [5,]     0  0.000000000  0.00000000     0     0     0     0  0.000000000     0
##      [,44] [,45] [,46] [,47] [,48] [,49]
## [1,]     0     0     0     0     0     0
## [2,]     0     0     0     0     0     0
## [3,]     0     0     0     0     0     0
## [4,]     0     0     0     0     0     0
## [5,]     0     0     0     0     0     0
## 
## $forest[[1]]$col_indices
##      [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14]
## [1,]    8   16   14    8   11   12    3    0    0     4     0     5     0     0
## [2,]    2   14   15   17    0    0   14    0    1    11    15    10     0     0
## [3,]   11    7    3    1   14   16   12    0    0     1     3    16     0     0
## [4,]    1    6    2   16    6   13    6    0    0     0    11     4     0     0
## [5,]    6    9   13    0    4    4   15    0    0     0    14    14     0     0
##      [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25] [,26]
## [1,]     0     0     0     0     0     4     0     1     0     1    15     9
## [2,]     0     0     0     0     0     8     0    13     0     8     9     8
## [3,]     0     0     0     0     0    13     0     9     0    13    12    15
## [4,]     0     0     0     0     0    11     0    17     0     2    16    14
## [5,]     0     0     0     0     0     6     0     0     0     9     6     5
##      [,27] [,28] [,29] [,30] [,31] [,32] [,33] [,34] [,35] [,36] [,37] [,38]
## [1,]    11    12     0     8    14     0     0    16     0     9     8     0
## [2,]     1    17     0    12    12     0     0     0     0    14    13     0
## [3,]     5    11     0     1     8     0     0     4     0     0     0     0
## [4,]    12     0     0     0     0     0     0     0     0     0     0     0
## [5,]    13    13     0     0     0     0     0     0     0     0     0     0
##      [,39] [,40] [,41] [,42] [,43] [,44] [,45] [,46] [,47] [,48] [,49]
## [1,]     0     0     0    10     0     0     0     0     0     0     0
## [2,]     0     0     0     9     0     0     0     0     0     0     0
## [3,]     0     0     0     0     0     0     0     0     0     0     0
## [4,]     0     0     0     0     0     0     0     0     0     0     0
## [5,]     0     0     0     0     0     0     0     0     0     0     0
## 
## $forest[[1]]$cut_points
##  [1]  4.12089249  9.11241423  3.35725528  1.13029653 -0.06338726  1.83650418
##  [7] -2.31065152  0.00000000 -4.73905557  3.01329103 -0.42466841 -0.82499625
## [13]  0.00000000  0.00000000  0.00000000  0.00000000  0.00000000  0.00000000
## [19]  0.00000000  0.98365264  0.00000000  0.30984580  0.00000000  1.53738164
## [25] 16.35000609 -0.27841756 -6.74177416 -0.83251714  0.00000000  8.64789556
## [31]  1.64580492  0.00000000  0.00000000  1.61411684  0.00000000 -1.89815296
## [37] -1.76669261  0.00000000  0.00000000  0.00000000  0.00000000 -2.76641769
## [43]  0.00000000  0.00000000  0.00000000  0.00000000  0.00000000  0.00000000
## [49]  0.00000000
## 
## $forest[[1]]$children_left
##       [,1]
##  [1,]    1
##  [2,]    3
##  [3,]    5
##  [4,]    7
##  [5,]    9
##  [6,]   11
##  [7,]   13
##  [8,]    0
##  [9,]   15
## [10,]   17
## [11,]   19
## [12,]   21
## [13,]    0
## [14,]    0
## [15,]    0
## [16,]    0
## [17,]    0
## [18,]    0
## [19,]    0
## [20,]   23
## [21,]    0
## [22,]   25
## [23,]    0
## [24,]   27
## [25,]   29
## [26,]   31
## [27,]   33
## [28,]   35
## [29,]    0
## [30,]   37
## [31,]   39
## [32,]    0
## [33,]    0
## [34,]   41
## [35,]    0
## [36,]   43
## [37,]   45
## [38,]    0
## [39,]    0
## [40,]    0
## [41,]    0
## [42,]   47
## [43,]    0
## [44,]    0
## [45,]    0
## [46,]    0
## [47,]    0
## [48,]    0
## [49,]    0
## 
## $forest[[1]]$rows_oobag
##       [,1]
##  [1,]    2
##  [2,]    3
##  [3,]    4
##  [4,]    6
##  [5,]   11
##  [6,]   16
##  [7,]   21
##  [8,]   23
##  [9,]   25
## [10,]   29
## [11,]   30
## [12,]   31
## [13,]   32
## [14,]   34
## [15,]   35
## [16,]   39
## [17,]   41
## [18,]   42
## [19,]   46
## [20,]   47
## [21,]   49
## [22,]   52
## [23,]   62
## [24,]   68
## [25,]   70
## [26,]   72
## [27,]   73
## [28,]   80
## [29,]   83
## [30,]   87
## [31,]   88
## [32,]   98
## [33,]  104
## [34,]  105
## [35,]  107
## [36,]  108
## [37,]  114
## [38,]  115
## [39,]  120
## [40,]  122
## [41,]  123
## [42,]  124
## [43,]  130
## [44,]  133
## [45,]  137
## [46,]  142
## [47,]  151
## [48,]  155
## [49,]  156
## [50,]  159
## [51,]  162
## [52,]  166
## [53,]  168
## [54,]  169
## [55,]  171
## [56,]  172
## [57,]  174
## [58,]  175
## [59,]  176
## [60,]  177
## [61,]  180
## [62,]  182
## [63,]  184
## [64,]  185
## [65,]  192
## [66,]  196
## [67,]  199
## [68,]  200
## [69,]  204
## [70,]  205
## [71,]  208
## [72,]  210
## [73,]  211
## [74,]  212
## [75,]  214
## [76,]  215
## [77,]  221
## [78,]  223
## [79,]  225
## [80,]  227
## [81,]  228
## [82,]  230
## [83,]  236
## [84,]  237
## [85,]  239
## [86,]  241
## [87,]  243
## [88,]  245
## [89,]  248
## [90,]  250
## [91,]  261
## [92,]  263
## [93,]  269
## [94,]  271
## [95,]  275
## 
## 
## 
## $surv_oobag
##             [,1]
##   [1,] 0.0000000
##   [2,] 0.0000000
##   [3,] 0.0000000
##   [4,] 0.0000000
##   [5,] 0.0000000
##   [6,] 0.9861111
##   [7,] 0.0000000
##   [8,] 0.0000000
##   [9,] 0.0000000
##  [10,] 0.0000000
##  [11,] 0.0000000
##  [12,] 1.0000000
##  [13,] 0.0000000
##  [14,] 0.0000000
##  [15,] 1.0000000
##  [16,] 0.0000000
##  [17,] 0.0000000
##  [18,] 0.0000000
##  [19,] 1.0000000
##  [20,] 0.0000000
##  [21,] 0.0000000
##  [22,] 0.0000000
##  [23,] 1.0000000
##  [24,] 1.0000000
##  [25,] 0.0000000
##  [26,] 0.0000000
##  [27,] 0.0000000
##  [28,] 0.0000000
##  [29,] 0.0000000
##  [30,] 0.0000000
##  [31,] 0.0000000
##  [32,] 0.0000000
##  [33,] 1.0000000
##  [34,] 0.9861111
##  [35,] 0.0000000
##  [36,] 0.0000000
##  [37,] 0.9861111
##  [38,] 0.5000000
##  [39,] 0.0000000
##  [40,] 0.0000000
##  [41,] 0.9861111
##  [42,] 0.4000000
##  [43,] 0.0000000
##  [44,] 0.0000000
##  [45,] 0.0000000
##  [46,] 0.0000000
##  [47,] 0.0000000
##  [48,] 0.0000000
##  [49,] 0.0000000
##  [50,] 1.0000000
##  [51,] 0.0000000
##  [52,] 1.0000000
##  [53,] 1.0000000
##  [54,] 0.0000000
##  [55,] 1.0000000
##  [56,] 0.0000000
##  [57,] 0.0000000
##  [58,] 0.0000000
##  [59,] 0.0000000
##  [60,] 1.0000000
##  [61,] 0.0000000
##  [62,] 0.0000000
##  [63,] 0.0000000
##  [64,] 0.0000000
##  [65,] 0.0000000
##  [66,] 1.0000000
##  [67,] 1.0000000
##  [68,] 0.0000000
##  [69,] 0.2000000
##  [70,] 1.0000000
##  [71,] 0.0000000
##  [72,] 0.0000000
##  [73,] 1.0000000
##  [74,] 0.0000000
##  [75,] 1.0000000
##  [76,] 0.0000000
##  [77,] 0.0000000
##  [78,] 0.0000000
##  [79,] 0.5000000
##  [80,] 0.0000000
##  [81,] 1.0000000
##  [82,] 0.0000000
##  [83,] 0.0000000
##  [84,] 0.0000000
##  [85,] 0.0000000
##  [86,] 0.0000000
##  [87,] 0.0000000
##  [88,] 1.0000000
##  [89,] 0.0000000
##  [90,] 0.9861111
##  [91,] 0.2000000
##  [92,] 0.0000000
##  [93,] 0.0000000
##  [94,] 0.0000000
##  [95,] 0.9861111
##  [96,] 0.0000000
##  [97,] 1.0000000
##  [98,] 0.0000000
##  [99,] 0.0000000
## [100,] 0.0000000
## [101,] 0.0000000
## [102,] 0.0000000
## [103,] 0.0000000
## [104,] 0.0000000
## [105,] 0.2500000
## [106,] 0.0000000
## [107,] 0.0000000
## [108,] 0.0000000
## [109,] 0.0000000
## [110,] 1.0000000
## [111,] 0.0000000
## [112,] 0.0000000
## [113,] 0.0000000
## [114,] 0.0000000
## [115,] 0.0000000
## [116,] 1.0000000
## [117,] 0.0000000
## [118,] 1.0000000
## [119,] 0.9861111
## [120,] 0.0000000
## [121,] 0.0000000
## [122,] 0.0000000
## [123,] 0.0000000
## [124,] 0.9861111
## [125,] 0.9861111
## [126,] 0.0000000
## [127,] 1.0000000
## [128,] 0.0000000
## [129,] 0.0000000
## [130,] 0.0000000
## [131,] 0.8750000
## [132,] 0.0000000
## [133,] 0.0000000
## [134,] 0.9861111
## [135,] 0.8750000
## [136,] 0.9861111
## [137,] 0.0000000
## [138,] 0.0000000
## [139,] 1.0000000
## [140,] 0.0000000
## [141,] 0.9861111
## [142,] 1.0000000
## [143,] 0.0000000
## [144,] 0.0000000
## [145,] 0.0000000
## [146,] 1.0000000
## [147,] 1.0000000
## [148,] 0.0000000
## [149,] 0.0000000
## [150,] 0.9861111
## [151,] 0.9861111
## [152,] 0.0000000
## [153,] 0.0000000
## [154,] 0.0000000
## [155,] 0.0000000
## [156,] 0.9861111
## [157,] 0.9861111
## [158,] 0.0000000
## [159,] 0.0000000
## [160,] 0.0000000
## [161,] 0.0000000
## [162,] 0.0000000
## [163,] 0.0000000
## [164,] 0.0000000
## [165,] 0.0000000
## [166,] 0.4000000
## [167,] 0.0000000
## [168,] 0.0000000
## [169,] 0.0000000
## [170,] 0.9861111
## [171,] 0.9861111
## [172,] 0.0000000
## [173,] 1.0000000
## [174,] 0.0000000
## [175,] 0.9861111
## [176,] 0.0000000
## [177,] 0.0000000
## [178,] 0.9861111
## [179,] 0.0000000
## [180,] 0.0000000
## [181,] 0.0000000
## [182,] 0.9861111
## [183,] 0.0000000
## [184,] 1.0000000
## [185,] 0.0000000
## [186,] 0.0000000
## [187,] 0.0000000
## [188,] 0.0000000
## [189,] 0.9861111
## [190,] 0.0000000
## [191,] 0.0000000
## [192,] 0.0000000
## [193,] 0.0000000
## [194,] 0.0000000
## [195,] 0.0000000
## [196,] 0.9861111
## [197,] 0.0000000
## [198,] 0.0000000
## [199,] 0.0000000
## [200,] 0.0000000
## [201,] 0.0000000
## [202,] 0.0000000
## [203,] 0.0000000
## [204,] 0.0000000
## [205,] 0.0000000
## [206,] 1.0000000
## [207,] 0.0000000
## [208,] 0.0000000
## [209,] 0.0000000
## [210,] 0.0000000
## [211,] 0.0000000
## [212,] 0.0000000
## [213,] 0.0000000
## [214,] 1.0000000
## [215,] 0.0000000
## [216,] 0.0000000
## [217,] 0.0000000
## [218,] 0.0000000
## [219,] 1.0000000
## [220,] 0.0000000
## [221,] 0.2500000
## [222,] 0.0000000
## [223,] 0.0000000
## [224,] 0.0000000
## [225,] 0.0000000
## [226,] 1.0000000
## [227,] 0.0000000
## [228,] 1.0000000
## [229,] 0.2000000
## [230,] 0.0000000
## [231,] 0.0000000
## [232,] 0.0000000
## [233,] 0.0000000
## [234,] 0.2000000
## [235,] 0.0000000
## [236,] 1.0000000
## [237,] 0.9861111
## [238,] 0.0000000
## [239,] 0.0000000
## [240,] 1.0000000
## [241,] 0.9861111
## [242,] 0.0000000
## [243,] 1.0000000
## [244,] 0.0000000
## [245,] 0.0000000
## [246,] 0.0000000
## [247,] 0.9861111
## [248,] 0.0000000
## [249,] 0.0000000
## [250,] 0.0000000
## [251,] 0.0000000
## [252,] 0.0000000
## [253,] 0.0000000
## [254,] 0.0000000
## [255,] 0.0000000
## [256,] 0.0000000
## [257,] 0.0000000
## [258,] 0.0000000
## [259,] 0.0000000
## [260,] 1.0000000
## [261,] 0.0000000
## [262,] 1.0000000
## [263,] 0.0000000
## [264,] 0.0000000
## [265,] 1.0000000
## [266,] 0.0000000
## [267,] 0.0000000
## [268,] 0.0000000
## [269,] 0.0000000
## [270,] 0.9861111
## [271,] 0.0000000
## [272,] 0.0000000
## [273,] 0.0000000
## [274,] 0.0000000
## [275,] 0.9861111
## [276,] 0.0000000
## 
## $pred_horizon
## [1] 1788
## 
## $eval_oobag
## $eval_oobag$stat_values
##           [,1]
## [1,] 0.5159148
## 
## $eval_oobag$stat_type
## [1] "Harrell's C-statistic"
## 
## 
## $importance
##     edema_1   ascites_1 trt_placebo         age        bili     protime 
##  1.00000000  0.66666667  0.27272727  0.22222222  0.21428571  0.20000000 
##    hepato_1         ast      copper    platelet    alk.phos        chol 
##  0.20000000  0.18181818  0.16666667  0.14285714  0.11111111  0.07142857 
##       stage        trig     albumin   edema_0.5   spiders_1       sex_f 
##  0.00000000  0.00000000  0.00000000  0.00000000  0.00000000  0.00000000 
## 
## $data
##     time status             trt      age sex ascites hepato spiders edema bili
## 1    400      1 d_penicill_main 58.76523   f       1      1       1     1 14.5
## 2   4500      0 d_penicill_main 56.44627   f       0      1       1     0  1.1
## 3   1012      1 d_penicill_main 70.07255   m       0      0       0   0.5  1.4
## 4   1925      1 d_penicill_main 54.74059   f       0      1       1   0.5  1.8
## 5   1504      0         placebo 38.10541   f       0      1       1     0  3.4
## 7   1832      0         placebo 55.53457   f       0      1       0     0  1.0
## 8   2466      1         placebo 53.05681   f       0      0       0     0  0.3
## 9   2400      1 d_penicill_main 42.50787   f       0      0       1     0  3.2
## 10    51      1         placebo 70.55989   f       1      0       1     1 12.6
## 11  3762      1         placebo 53.71389   f       0      1       1     0  1.4
## 12   304      1         placebo 59.13758   f       0      0       1     0  3.6
## 13  3577      0         placebo 45.68925   f       0      0       0     0  0.7
## 15  3584      1 d_penicill_main 64.64613   f       0      0       0     0  0.8
## 16  3672      0         placebo 40.44353   f       0      0       0     0  0.7
## 17   769      1         placebo 52.18344   f       0      1       0     0  2.7
## 18   131      1 d_penicill_main 53.93018   f       0      1       1     1 11.4
## 19  4232      0 d_penicill_main 49.56057   f       0      1       0   0.5  0.7
## 20  1356      1         placebo 59.95346   f       0      1       0     0  5.1
## 21  3445      0         placebo 64.18891   m       0      1       1     0  0.6
## 22   673      1 d_penicill_main 56.27652   f       0      0       1     0  3.4
## 23   264      1         placebo 55.96715   f       1      1       1     1 17.4
## 24  4079      1 d_penicill_main 44.52019   m       0      1       0     0  2.1
## 25  4127      0         placebo 45.07324   f       0      0       0     0  0.7
## 26  1444      1         placebo 52.02464   f       0      1       1     0  5.2
## 27    77      1         placebo 54.43943   f       1      1       1   0.5 21.6
## 28   549      1         placebo 44.94730   f       1      1       1     1 17.2
## 29  4509      0         placebo 63.87680   f       0      0       0     0  0.7
## 30   321      1         placebo 41.38535   f       0      1       1     0  3.6
## 31  3839      1         placebo 41.55236   f       0      1       0     0  4.7
## 32  4523      0         placebo 53.99589   f       0      1       0     0  1.8
## 33  3170      1         placebo 51.28268   f       0      0       0     0  0.8
## 34  3933      0 d_penicill_main 52.06023   f       0      0       0     0  0.8
## 35  2847      1         placebo 48.61875   f       0      0       0     0  1.2
## 36  3611      0         placebo 56.41068   f       0      0       0     0  0.3
## 37   223      1 d_penicill_main 61.72758   f       1      1       0     1  7.1
## 38  3244      1         placebo 36.62697   f       0      1       1     0  3.3
## 39  2297      1 d_penicill_main 55.39220   f       0      1       0     0  0.7
## 43  4556      0 d_penicill_main 48.87064   f       0      0       0     0  1.1
## 44  3428      1         placebo 37.58248   f       0      1       1     1  3.3
## 46  2256      1 d_penicill_main 45.79877   f       0      1       0     0  5.7
## 47  2576      0         placebo 47.42779   f       0      0       0     0  0.5
## 48  4427      0         placebo 49.13621   m       0      0       0     0  1.9
## 50  2598      1 d_penicill_main 53.50856   f       0      1       0     0  1.1
## 51  3853      1         placebo 52.08761   f       0      0       0     0  0.8
## 52  2386      1 d_penicill_main 50.54073   m       0      0       0     0  6.0
## 54  1434      1 d_penicill_main 39.19781   f       1      1       1     1  1.3
## 55  1360      1 d_penicill_main 65.76318   m       0      0       0     0  1.8
## 56  1847      1         placebo 33.61807   f       0      1       1     0  1.1
## 57  3282      1 d_penicill_main 53.57153   f       0      1       0   0.5  2.3
## 59  2224      1 d_penicill_main 40.39425   f       0      1       1     0  0.8
## 60  4365      0 d_penicill_main 58.38193   f       0      0       0     0  0.9
## 61  4256      0         placebo 43.89870   m       0      0       0     0  0.6
## 62  3090      1         placebo 60.70637   f       1      0       0     0  1.3
## 63   859      1         placebo 46.62834   f       0      0       1     1 22.5
## 64  1487      1         placebo 62.90760   f       0      1       0     0  2.1
## 65  3992      0 d_penicill_main 40.20260   f       0      0       0     0  1.2
## 66  4191      1 d_penicill_main 46.45311   m       0      1       0     0  1.4
## 67  2769      1         placebo 51.28816   f       0      0       0     0  1.1
## 68  4039      0 d_penicill_main 32.61328   f       0      0       0     0  0.7
## 69  1170      1 d_penicill_main 49.33881   f       0      1       1   0.5 20.0
## 71  4196      0         placebo 48.84600   f       0      1       0     0  1.2
## 72  4184      0         placebo 32.49281   f       0      0       0     0  0.5
## 73  4190      0         placebo 38.49418   f       0      0       0     0  0.7
## 74  1827      1 d_penicill_main 51.92060   f       0      1       1     0  8.4
## 75  1191      1 d_penicill_main 43.51814   f       1      1       1   0.5 17.1
## 76    71      1 d_penicill_main 51.94251   f       0      1       1   0.5 12.2
## 77   326      1         placebo 49.82615   f       0      1       1   0.5  6.6
## 78  1690      1 d_penicill_main 47.94524   f       0      1       0     0  6.3
## 79  3707      0 d_penicill_main 46.51608   f       0      1       0     0  0.8
## 80   890      1         placebo 67.41136   m       0      1       0     0  7.2
## 81  2540      1 d_penicill_main 63.26352   f       0      1       1     0 14.4
## 82  3574      1 d_penicill_main 67.31006   f       0      0       0     0  4.5
## 83  4050      0 d_penicill_main 56.01369   f       0      1       0   0.5  1.3
## 84  4032      0         placebo 55.83025   f       0      0       0     0  0.4
## 85  3358      1         placebo 47.21697   f       0      1       0     0  2.1
## 86  1657      1 d_penicill_main 52.75838   f       0      1       1     0  5.0
## 87   198      1 d_penicill_main 37.27858   f       0      0       0     0  1.1
## 88  2452      0         placebo 41.39357   f       0      0       0   0.5  0.6
## 89  1741      1 d_penicill_main 52.44353   f       0      1       0     0  2.0
## 90  2689      1 d_penicill_main 33.47570   m       0      0       0     0  1.6
## 91   460      1         placebo 45.60712   f       0      1       1   0.5  5.0
## 92   388      1 d_penicill_main 76.70910   f       1      0       0     1  1.4
## 93  3913      0 d_penicill_main 36.53388   f       0      0       0     0  1.3
## 94   750      1 d_penicill_main 53.91650   f       0      1       1     0  3.2
## 97   611      1         placebo 71.89322   m       0      1       0   0.5  2.0
## 98  3823      0 d_penicill_main 28.88433   f       0      0       0     0  1.0
## 99  3820      0         placebo 48.46817   m       0      0       0     0  1.8
## 100  552      1         placebo 51.46886   m       0      1       0     0  2.3
## 101 3581      0         placebo 44.95003   f       0      0       0     0  0.9
## 102 3099      0 d_penicill_main 56.56947   f       0      0       0     0  0.9
## 103  110      1         placebo 48.96372   f       1      1       1     1  2.5
## 104 3086      1 d_penicill_main 43.01711   f       0      0       0     0  1.1
## 105 3092      0         placebo 34.03970   f       0      1       0     0  1.1
## 107 3388      0         placebo 62.52156   f       0      0       0     0  0.6
## 108 2583      1 d_penicill_main 50.35729   f       0      0       0     0  0.4
## 109 2504      0         placebo 44.06297   f       0      0       0     0  0.5
## 110 2105      1 d_penicill_main 38.91034   f       0      1       1     0  1.9
## 111 2350      0 d_penicill_main 41.15264   f       0      0       0     0  5.5
## 112 3445      1         placebo 55.45791   f       0      1       1     0  2.0
## 113  980      1 d_penicill_main 51.23340   f       0      1       1     0  6.7
## 114 3395      1         placebo 52.82683   m       0      0       0     0  3.2
## 115 3422      0         placebo 42.63929   f       0      0       1     0  0.7
## 116 3336      0 d_penicill_main 61.07050   f       0      0       1   0.5  3.0
## 117 1083      1 d_penicill_main 49.65640   f       0      1       1     0  6.5
## 118 2288      1 d_penicill_main 48.85421   f       0      1       0     0  3.5
## 119  515      1 d_penicill_main 54.25599   f       0      0       1     0  0.6
## 120 2033      0 d_penicill_main 35.15127   m       0      0       0     0  3.5
## 121  191      1         placebo 67.90691   m       1      1       0     1  1.3
## 122 3297      0 d_penicill_main 55.43600   f       0      0       0     0  0.6
## 124 3069      0 d_penicill_main 52.88980   m       0      1       0     0  0.6
## 125 2468      0         placebo 47.18138   f       0      1       0     0  1.3
## 127 3255      0         placebo 44.10404   f       0      0       0     0  0.5
## 130 1413      1         placebo 44.22724   f       0      1       1     0 17.4
## 131  850      1         placebo 62.00137   f       0      1       1     0  2.8
## 132 2944      0 d_penicill_main 40.55305   f       0      0       0     0  1.9
## 133 2796      1         placebo 62.64476   m       0      0       0     0  1.5
## 134 3149      0         placebo 42.33539   f       0      0       0     0  0.7
## 135 3150      0 d_penicill_main 42.96783   f       0      0       0     0  0.4
## 136 3098      0 d_penicill_main 55.96167   f       0      0       0     0  0.8
## 137 2990      0 d_penicill_main 62.86105   f       0      0       0     0  1.1
## 138 1297      1 d_penicill_main 51.24983   m       0      1       0     0  7.3
## 139 2106      0         placebo 46.76249   f       0      1       0     0  1.1
## 140 3059      0 d_penicill_main 54.07529   f       0      1       0     0  1.1
## 141 3050      0 d_penicill_main 47.03628   f       0      0       0     0  0.9
## 142 2419      1         placebo 55.72621   f       0      1       0     0  1.0
## 143  786      1         placebo 46.10267   f       0      1       0     0  2.9
## 144  943      1         placebo 52.28747   f       0      1       0   0.5 28.0
## 145 2976      0         placebo 51.20055   f       0      0       1     0  0.7
## 147 2995      0 d_penicill_main 75.01164   f       0      0       0   0.5  1.2
## 148 1427      1         placebo 30.86379   f       0      1       0     0  7.2
## 149  762      1 d_penicill_main 61.80424   m       0      1       1   0.5  3.0
## 151 2870      0 d_penicill_main 55.04175   f       0      0       0     0  0.9
## 152 1152      1 d_penicill_main 69.94114   m       0      1       0     0  2.3
## 153 2863      0 d_penicill_main 49.60438   f       0      0       0     0  0.5
## 154  140      1 d_penicill_main 69.37714   m       0      0       1     1  2.4
## 155 2666      0         placebo 43.55647   f       0      1       1   0.5  0.6
## 156  853      1         placebo 59.40862   f       0      1       0     0 25.5
## 157 2835      0         placebo 48.75838   f       0      0       0     0  0.6
## 158 2475      0 d_penicill_main 36.49281   f       0      0       0     0  3.4
## 159 1536      1         placebo 45.76044   m       0      0       0     0  2.5
## 160 2772      0         placebo 57.37166   f       0      0       0     0  0.6
## 161 2797      0         placebo 42.74333   f       0      0       0     0  2.3
## 162  186      1         placebo 58.81725   f       0      1       1     0  3.2
## 163 2055      1 d_penicill_main 53.49760   f       0      0       0     0  0.3
## 165 1077      1 d_penicill_main 53.30595   m       0      1       0     0  4.0
## 166 2721      0         placebo 41.35524   f       0      1       0     0  5.7
## 167 1682      1 d_penicill_main 60.95825   m       0      1       0     0  0.9
## 169 1212      1         placebo 35.49076   f       0      0       0     0  1.3
## 170 2692      0 d_penicill_main 48.66256   f       0      0       0     0  1.2
## 172 2301      0         placebo 49.86995   f       0      0       1     0  1.3
## 173 2657      0 d_penicill_main 30.27515   f       0      1       1     0  3.0
## 175 2624      0         placebo 52.15332   f       0      0       0     0  0.8
## 177 2609      0         placebo 55.45243   f       0      0       0     0  0.9
## 179 2573      0         placebo 43.94251   f       0      1       0     0  1.8
## 180 2563      0         placebo 42.56810   f       0      0       0     0  4.7
## 181 2556      0 d_penicill_main 44.56947   f       0      1       1     0  1.4
## 183 2241      0         placebo 40.26010   f       0      0       0     0  0.5
## 184  974      1         placebo 37.60712   f       0      1       0     0 11.0
## 185 2527      0 d_penicill_main 48.36140   f       0      0       0     0  0.8
## 186 1576      1 d_penicill_main 70.83641   f       0      0       1   0.5  2.0
## 187  733      1         placebo 35.79192   f       0      1       0     0 14.0
## 188 2332      0 d_penicill_main 62.62286   f       0      1       0     0  0.7
## 189 2456      0         placebo 50.64750   f       0      1       0     0  1.3
## 191  216      1         placebo 52.69268   f       1      1       1     0 24.5
## 192 2443      0 d_penicill_main 52.72005   f       0      1       0     0  0.9
## 193  797      1         placebo 56.77207   f       0      0       0     0 10.8
## 194 2449      0 d_penicill_main 44.39699   f       0      0       0     0  1.5
## 195 2330      0 d_penicill_main 29.55510   f       0      1       0     0  3.7
## 196 2363      0 d_penicill_main 57.04038   f       0      1       1     0  1.4
## 197 2365      0 d_penicill_main 44.62697   f       0      0       0     0  0.6
## 198 2357      0         placebo 35.79740   f       0      0       1     0  0.7
## 199 1592      0 d_penicill_main 40.71732   f       0      0       0     0  2.1
## 200 2318      0         placebo 32.23272   f       0      0       1     0  4.7
## 201 2294      0         placebo 41.09240   f       0      1       0     0  0.6
## 202 2272      0 d_penicill_main 61.63997   f       0      0       0     0  0.5
## 203 2221      0         placebo 37.05681   f       0      1       0     0  0.5
## 204 2090      1         placebo 62.57906   f       0      0       0     0  0.7
## 206 2255      0 d_penicill_main 61.99042   f       0      0       0     0  0.6
## 208  904      1 d_penicill_main 61.29500   f       0      1       0     0  3.9
## 209 2216      0         placebo 52.62423   f       0      1       1     0  0.7
## 210 2224      0         placebo 49.76318   m       0      1       0     0  0.9
## 212 2176      0         placebo 47.26352   f       0      0       0     0  1.2
## 213 2178      0 d_penicill_main 50.20397   f       0      0       1     0  0.5
## 214 1786      1         placebo 69.34702   f       0      1       0     0  0.9
## 215 1080      1         placebo 41.16906   f       0      0       0     0  5.9
## 217  790      1         placebo 36.07940   f       0      1       0     0 11.4
## 219 2157      0         placebo 42.71321   f       0      0       0     0  1.6
## 220 1235      1 d_penicill_main 63.63039   f       0      0       1     0  3.8
## 221 2050      0         placebo 56.62971   f       0      1       0     0  0.9
## 222  597      1         placebo 46.26420   f       0      1       0     0  4.5
## 223  334      1 d_penicill_main 61.24298   f       1      1       0     1 14.1
## 224 1945      0 d_penicill_main 38.62012   f       0      0       0     0  1.0
## 225 2022      0 d_penicill_main 38.77070   f       0      0       0     0  0.7
## 226 1978      0         placebo 56.69541   f       0      1       0     0  0.5
## 227  999      1 d_penicill_main 58.95140   m       0      0       0     0  2.3
## 228 1967      0         placebo 36.92266   f       0      0       0     0  0.7
## 229  348      1 d_penicill_main 62.41478   f       1      1       0   0.5  4.5
## 230 1979      0         placebo 34.60917   f       0      1       1     0  3.3
## 231 1165      1         placebo 58.33539   f       0      1       1     0  3.4
## 232 1951      0 d_penicill_main 50.18207   f       0      1       0     0  0.4
## 233 1932      0 d_penicill_main 42.68583   f       0      1       1     0  0.9
## 234 1776      0         placebo 34.37919   f       0      0       0     0  0.9
## 235 1882      0         placebo 33.18275   f       0      1       0     0 13.0
## 236 1908      0 d_penicill_main 38.38193   f       0      1       1     0  1.5
## 237 1882      0 d_penicill_main 59.76181   f       0      1       0     0  1.6
## 239  694      1 d_penicill_main 46.78987   f       0      1       1     0  0.8
## 240 1831      0 d_penicill_main 56.07940   f       0      0       0     0  0.4
## 241  837      0         placebo 41.37440   f       0      1       1     0  4.4
## 242 1810      0 d_penicill_main 64.57221   f       0      1       0     0  1.9
## 243  930      1         placebo 67.48802   f       0      1       0     0  8.0
## 244 1690      1 d_penicill_main 44.82957   f       0      0       1     0  3.9
## 245 1790      0         placebo 45.77139   f       0      1       0     0  0.6
## 246 1435      0 d_penicill_main 32.95003   f       0      1       0     0  2.1
## 247  732      0 d_penicill_main 41.22108   f       0      1       0     0  6.1
## 248 1785      0         placebo 55.41684   f       0      1       0     0  0.8
## 249 1783      0 d_penicill_main 47.98084   f       0      0       1     0  1.3
## 250 1769      0         placebo 40.79124   f       0      1       0     0  0.6
## 251 1457      0 d_penicill_main 56.97467   f       0      0       0     0  0.5
## 252 1770      0 d_penicill_main 68.46270   f       0      1       1     0  1.1
## 253 1765      0 d_penicill_main 78.43943   m       1      1       1     0  7.1
## 254  737      0 d_penicill_main 39.85763   f       0      1       1     0  3.1
## 255 1735      0         placebo 35.31006   f       0      1       1     0  0.7
## 256 1701      0 d_penicill_main 31.44422   f       0      0       0     0  1.1
## 257 1614      0 d_penicill_main 58.26420   f       0      0       0     0  0.5
## 258 1702      0 d_penicill_main 51.48802   f       0      0       0     0  1.1
## 259 1615      0         placebo 59.96988   f       0      1       0     0  3.1
## 260 1656      0         placebo 74.52430   m       0      1       0     0  5.6
## 262 1666      0         placebo 42.78713   f       0      1       0     0  2.8
## 263 1301      0         placebo 34.87474   f       0      1       1   0.5  1.1
## 264 1542      0         placebo 44.13963   f       0      1       1     0  3.4
## 265 1084      0         placebo 46.38193   f       0      1       0     0  3.5
## 266 1614      0 d_penicill_main 56.30938   f       0      0       0     0  0.5
## 267  179      1 d_penicill_main 70.90760   f       1      1       1     1  6.6
## 268 1191      1 d_penicill_main 55.39493   f       1      1       0   0.5  6.4
## 269 1363      0         placebo 45.08419   f       0      0       0     0  3.6
## 270 1568      0 d_penicill_main 26.27789   f       0      1       1     0  1.0
## 271 1569      0         placebo 50.47228   f       0      1       0     0  1.0
## 272 1525      0 d_penicill_main 38.39836   f       0      0       0     0  0.5
## 273 1558      0         placebo 47.41958   f       0      0       1     0  2.2
## 275 1349      0 d_penicill_main 38.31622   f       0      0       0     0  2.2
## 276 1481      0 d_penicill_main 50.10815   f       0      0       0     0  1.0
## 277 1434      0         placebo 35.08830   f       0      0       0   0.5  1.0
## 278 1420      0         placebo 32.50376   f       0      0       0     0  5.6
## 279 1433      0         placebo 56.15332   f       0      0       0     0  0.5
## 280 1412      0 d_penicill_main 46.15469   f       0      0       0     0  1.6
## 281   41      1 d_penicill_main 65.88364   f       1      0       0     1 17.9
## 282 1455      0         placebo 33.94387   f       0      1       0     0  1.3
## 283 1030      0         placebo 62.86105   f       0      0       0     0  1.1
## 284 1418      0         placebo 48.56400   f       0      0       0     0  1.3
## 285 1401      0 d_penicill_main 46.34908   f       0      0       0     0  0.8
## 286 1408      0 d_penicill_main 38.85284   f       0      1       1     0  2.0
## 287 1234      0 d_penicill_main 58.64750   f       0      0       1     0  6.4
## 288 1067      0         placebo 48.93634   f       0      1       0   0.5  8.7
## 289  799      1 d_penicill_main 67.57290   m       0      1       0   0.5  4.0
## 290 1363      0 d_penicill_main 65.98494   f       0      0       0     0  1.4
## 291  901      0 d_penicill_main 40.90075   f       0      0       0     0  3.2
## 292 1329      0         placebo 50.24504   m       0      1       0     0  8.6
## 293 1320      0         placebo 57.19644   f       0      1       1     1  8.5
## 294 1302      0 d_penicill_main 60.53662   m       0      1       0     0  6.6
## 295  877      0 d_penicill_main 35.35113   m       0      0       0     0  2.4
## 296 1321      0         placebo 31.38125   f       0      0       0     0  0.8
## 297  533      0 d_penicill_main 55.98631   m       0      1       0     0  1.2
## 298 1300      0         placebo 52.72553   f       0      1       0     0  1.1
## 299 1293      0 d_penicill_main 38.09172   f       0      0       0     0  2.4
## 301 1295      0         placebo 45.21013   f       0      0       0     0  1.0
## 302 1271      0 d_penicill_main 37.79877   f       0      0       0     0  0.7
## 303 1250      0         placebo 60.65982   f       0      1       1     0  1.0
## 304 1230      0 d_penicill_main 35.53457   f       0      0       0     0  0.5
## 305 1216      0         placebo 43.06639   f       0      1       1     0  2.9
## 306 1216      0         placebo 56.39151   f       0      1       0     0  0.6
## 307 1149      0         placebo 30.57358   f       0      0       0     0  0.8
## 308 1153      0 d_penicill_main 61.18275   f       0      1       0     0  0.4
## 309  994      0         placebo 58.29979   f       0      0       0     0  0.4
## 310  939      0 d_penicill_main 62.33265   f       0      0       0     0  1.7
## 311  839      0 d_penicill_main 37.99863   f       0      0       0     0  2.0
## 312  788      0         placebo 33.15264   f       0      0       1     0  6.4
##     chol albumin copper alk.phos    ast trig platelet protime stage
## 1    261    2.60    156   1718.0 137.95  172      190    12.2     4
## 2    302    4.14     54   7394.8 113.52   88      221    10.6     3
## 3    176    3.48    210    516.0  96.10   55      151    12.0     4
## 4    244    2.54     64   6121.8  60.63   92      183    10.3     4
## 5    279    3.53    143    671.0 113.15   72      136    10.9     3
## 7    322    4.09     52    824.0  60.45  213      204     9.7     3
## 8    280    4.00     52   4651.2  28.38  189      373    11.0     3
## 9    562    3.08     79   2276.0 144.15   88      251    11.0     2
## 10   200    2.74    140    918.0 147.25  143      302    11.5     4
## 11   259    4.16     46   1104.0  79.05   79      258    12.0     4
## 12   236    3.52     94    591.0  82.15   95       71    13.6     4
## 13   281    3.85     40   1181.0  88.35  130      244    10.6     3
## 15   231    3.87    173   9009.8 127.71   96      295    11.0     3
## 16   204    3.66     28    685.0  72.85   58      198    10.8     3
## 17   274    3.15    159   1533.0 117.80  128      224    10.5     4
## 18   178    2.80    588    961.0 280.55  200      283    12.4     4
## 19   235    3.56     39   1881.0  93.00  123      209    11.0     3
## 20   374    3.51    140   1919.0 122.45  135      322    13.0     4
## 21   252    3.83     41    843.0  65.10   83      336    11.4     4
## 22   271    3.63    464   1376.0 120.90   55      173    11.6     4
## 23   395    2.94    558   6064.8 227.04  191      214    11.7     4
## 24   456    4.00    124   5719.0 221.88  230       70     9.9     2
## 25   298    4.10     40    661.0 106.95   66      324    11.3     2
## 26  1128    3.68     53   3228.0 165.85  166      421     9.9     3
## 27   175    3.31    221   3697.4 101.91  168       80    12.0     4
## 28   222    3.23    209   1975.0 189.10  195      144    13.0     4
## 29   370    3.78     24   5833.0  73.53   86      390    10.6     2
## 30   260    2.54    172   7277.0 121.26  158      124    11.0     4
## 31   296    3.44    114   9933.2 206.40  101      195    10.3     2
## 32   262    3.34    101   7277.0  82.56  158      286    10.6     4
## 33   210    3.19     82   1592.0 218.55  113      180    12.0     3
## 34   364    3.70     37   1840.0 170.50   64      273    10.5     2
## 35   314    3.20    201  12258.8  72.24  151      431    10.6     3
## 36   172    3.39     18    558.0  71.30   96      311    10.6     2
## 37   334    3.01    150   6931.2 180.60  118      102    12.0     4
## 38   383    3.53    102   1234.0 137.95   87      234    11.0     4
## 39   282    3.00     52   9066.8  72.24  111      563    10.6     4
## 43   361    3.64     36   5430.2  67.08   89      203    10.6     2
## 44   299    3.55    131   1029.0 119.35   50      199    11.7     3
## 46   482    2.84    161  11552.0 136.74  165      518    12.7     3
## 47   316    3.65     68   1716.0 187.55   71      356     9.8     3
## 48   259    3.70    281  10396.8 188.34  178      214    11.0     3
## 50   257    3.36     43   1080.0 106.95   73      128    10.6     4
## 51   276    3.60     54   4332.0  99.33  143      273    10.6     2
## 52   614    3.70    158   5084.4 206.40   93      362    10.6     1
## 54   288    3.40    262   5487.2  73.53  125      254    11.0     4
## 55   416    3.94    121  10165.0  79.98  219      213    11.0     3
## 56   498    3.80     88  13862.4  95.46  319      365    10.6     2
## 57   260    3.18    231  11320.2 105.78   94      216    12.4     3
## 59   329    3.50     49   7622.8 126.42  124      321    10.6     3
## 60   604    3.40     82    876.0  71.30   58      228    10.3     3
## 61   216    3.94     28    601.0  60.45  188      211    13.0     1
## 62   302    2.75     58   1523.0  43.40  112      329    13.2     4
## 63   932    3.12     95   5396.0 244.90  133      165    11.6     3
## 64   373    3.50     52   1009.0 150.35  188      178    11.0     3
## 65   256    3.60     74    724.0 141.05  108      430    10.0     1
## 66   427    3.70    105   1909.0 182.90  171      123    11.0     3
## 67   466    3.91     84   1787.0 328.60  185      261    10.0     3
## 68   174    4.09     58    642.0  71.30   46      203    10.6     3
## 69   652    3.46    159   3292.0 215.45  184      227    12.4     3
## 71   258    3.57     79   2201.0 120.90   76      410    11.5     4
## 72   320    3.54     51   1243.0 122.45   80      225    10.0     3
## 73   132    3.60     17    423.0  49.60   56      265    11.0     1
## 74   558    3.99    280    967.0  89.90  309      278    11.0     4
## 75   674    2.53    207   2078.0 182.90  598      268    11.5     4
## 76   394    3.08    111   2132.0 155.00  243      165    11.6     4
## 77   244    3.41    199   1819.0 170.50   91      132    12.1     3
## 78   436    3.02     75   2176.0 170.50  104      236    10.6     4
## 79   315    4.24     13   1637.0 170.50   70      426    10.9     3
## 80   247    3.72    269   1303.0 176.70   91      360    11.2     4
## 81   448    3.65     34   1218.0  60.45  318      385    11.7     4
## 82   472    4.09    154   1580.0 117.80  272      412    11.1     3
## 83   250    3.50     48   1138.0  71.30  100       81    12.9     4
## 84   263    3.76     29   1345.0 137.95   74      181    11.2     3
## 85   262    3.48     58   2045.0  89.90   84      225    11.5     4
## 86  1600    3.21     75   2656.0  82.15  174      181    10.9     3
## 87   345    4.40     75   1860.0 218.55   72      447    10.7     3
## 88   296    4.06     37   1032.0  80.60   83      442    12.0     3
## 89   408    3.65     50   1083.0 110.05   98      200    11.4     2
## 90   660    4.22     94   1857.0 151.90  155      337    11.0     2
## 91   325    3.47    110   2460.0 246.45   56      430    11.9     4
## 92   206    3.13     36   1626.0  86.80   70      145    12.2     4
## 93   353    3.67     73   2039.0 232.50   68      380    11.1     2
## 94   201    3.11    178   1212.0 159.65   69      188    11.8     4
## 97   420    3.26     62   3196.0  77.50   91      344    11.4     3
## 98   239    3.77     77   1877.0  97.65  101      312    10.2     1
## 99   460    3.35    148   1472.0 108.50  118      172    10.2     2
## 100  178    3.00    145    746.0 178.25  122      119    12.0     4
## 101  400    3.60     31   1689.0 164.30  166      327    10.4     3
## 102  248    3.97    172    646.0  62.00   84      128    10.1     1
## 103  188    3.67     57   1273.0 119.35  102      110    11.1     4
## 104  303    3.64     20   2108.0 128.65   53      349    11.1     2
## 105  464    4.20     38   1644.0 151.90  102      348    10.3     3
## 107  212    4.03     10    648.0  71.30   77      316    17.1     1
## 108  127    3.50     14   1062.0  49.60   84      334    10.3     2
## 109  120    3.61     53    804.0 110.05   52      271    10.6     3
## 110  486    3.54     74   1052.0 108.50  109      141    10.9     3
## 111  528    4.18     77   2404.0 172.05   78      467    10.7     3
## 112  267    3.67     89    754.0 196.85   90      136    11.8     4
## 113  374    3.74    103    979.0 128.65  100      266    11.1     4
## 114  259    4.30    208   1040.0 110.05   78      268    11.7     3
## 115  303    4.19     81   1584.0 111.60  156      307    10.3     3
## 116  458    3.63     74   1588.0 106.95  382      438     9.9     3
## 117  950    3.11    111   2374.0 170.50  149      354    11.0     4
## 118  390    3.30     67    878.0 137.95   93      207    10.2     3
## 119  636    3.83    129    944.0  97.65  114      306     9.5     3
## 120  325    3.98    444    766.0 130.20  210      344    10.6     3
## 121  151    3.08     73   1112.0  46.50   49      213    13.2     4
## 122  298    4.13     29    758.0  65.10   85      256    10.7     3
## 124  251    3.90     25    681.0  57.35  107      182    10.8     4
## 125  316    3.51     75   1162.0 147.25  137      238    10.0     4
## 127  268    4.08      9   1174.0  86.80   95      453    10.0     2
## 130 1775    3.43    205   2065.0 165.85   97      418    11.5     3
## 131  242    3.80     74    614.0 136.40  104      121    13.2     4
## 132  448    3.83     60   1052.0 127.10  175      181     9.8     3
## 133  331    3.95     13    577.0 128.65   99      165    10.1     4
## 134  578    3.67     35   1353.0 127.10  105      427    10.7     2
## 135  263    3.57    123    836.0  74.40  121      445    11.0     2
## 136  263    3.35     27   1636.0 116.25   69      206     9.8     2
## 137  399    3.60     79   3472.0 155.00  152      344    10.1     2
## 138  426    3.93    262   2424.0 145.70  218      252    10.5     3
## 139  328    3.31    159   1260.0  94.55  134      142    11.6     4
## 140  290    4.09     38   2120.0 186.00  146      318    10.0     3
## 141  346    3.77     59    794.0 125.55   56      336    10.6     2
## 142  364    3.48     20    720.0 134.85   88      283     9.9     2
## 143  332    3.60     86   1492.0 134.85  103      277    11.0     4
## 144  556    3.26    152   3896.0 198.40  171      335    10.0     3
## 145  309    3.84     96    858.0  41.85  106      253    11.4     3
## 147  288    3.37     32    791.0  57.35  114      213    10.7     2
## 148 1015    3.26    247   3836.0 198.40  280      330     9.8     3
## 149  257    3.79    290   1664.0 102.30  112      140     9.9     4
## 151  460    3.03     57    721.0  85.25  174      301     9.4     2
## 152  586    3.01    243   2276.0 114.70  126      339    10.9     3
## 153  217    3.85     68    453.0  54.25   68      270    11.1     1
## 154  168    2.56    225   1056.0 120.90   75      108    14.1     3
## 155  220    3.35     57   1620.0 153.45   80      311    11.2     4
## 156  358    3.52    219   2468.0 201.50  205      151    11.5     2
## 157  286    3.42     34   1868.0  77.50  206      487    10.0     2
## 158  450    3.37     32   1408.0 116.25  118      313    11.2     2
## 159  317    3.46    217    714.0 130.20  140      207    10.1     3
## 160  217    3.62     13    414.0  75.95  119      224    10.5     3
## 161  502    3.56      4    964.0 120.90  180      269     9.6     2
## 162  260    3.19     91    815.0 127.10  101      160    12.0     4
## 163  233    4.08     20    622.0  66.65   68      358     9.9     3
## 165  196    3.45     80   2496.0 133.30  142      212    11.3     4
## 166 1480    3.26     84   1960.0 457.25  108      213     9.5     2
## 167  376    3.86    200   1015.0  83.70  154      238    10.3     4
## 169  408    4.22     67   1387.0 142.60  137      295    10.1     3
## 170  390    3.61     32   1509.0  88.35   52      263     9.0     3
## 172  205    3.34     65   1031.0  91.45  126      217     9.8     3
## 173  236    3.42     76   1403.0  89.90   86      493     9.8     2
## 175  283    3.80    152    718.0 108.50  168      340    10.1     3
## 177  258    4.01     49    559.0  43.40  133      277    10.4     2
## 179  396    3.83     39   2148.0 102.30  133      278     9.9     4
## 180  478    4.38     44   1629.0 237.15   76      175    10.4     3
## 181  248    3.58     63    554.0  75.95  106       79    10.3     4
## 183  201    3.73     44   1345.0  54.25  145      445    10.1     2
## 184  674    3.55    358   2412.0 167.40  140      471     9.8     3
## 185  256    3.54     42   1132.0  74.40   94      192    10.5     3
## 186  225    3.53     51    933.0  69.75   62      200    12.7     3
## 187  808    3.43    251   2870.0 153.45  137      268    11.5     3
## 188  187    3.48     41    654.0 120.90   98      164    11.0     4
## 189  360    3.63     52   1812.0  97.65  164      256     9.9     3
## 191 1092    3.35    233   3740.0 147.25  432      399    15.2     4
## 192  308    3.69     67    696.0  51.15  101      344     9.8     4
## 193  932    3.19    267   2184.0 161.20  157      382    10.4     4
## 194  293    4.30     50    975.0 125.55   56      336     9.1     2
## 195  347    3.90     76   2544.0 221.65   90      129    11.5     4
## 196  226    3.36     13    810.0  72.85   62      117    11.6     4
## 197  266    3.97     25   1164.0 102.30  102      201    10.1     2
## 198  286    2.90     38   1692.0 141.05   90      381     9.6     2
## 199  392    3.43     52   1395.0 184.45  194      328    10.2     3
## 200  236    3.55    112   1391.0 137.95  114      332     9.9     3
## 201  235    3.20     26   1758.0 106.95   67      228    10.8     4
## 202  223    3.80     15   1044.0  80.60   89      514    10.0     2
## 203  149    4.04    227    598.0  52.70   57      166     9.9     2
## 204  255    3.74     23   1024.0  77.50   58      281    10.2     3
## 206  213    4.07     12   5300.0  57.35   68      240    11.0     1
## 208  396    3.20     58   1440.0 153.45  131      156    10.0     4
## 209  252    4.01     11   1210.0  72.85   58      309     9.5     2
## 210  346    3.37     81   1098.0 122.45   90      298    10.0     2
## 212  232    3.98     11   1074.0 100.75   99      223     9.9     3
## 213  400    3.40      9   1134.0  96.10   55      356    10.2     3
## 214  404    3.43     34   1866.0  79.05  224      236     9.9     3
## 215 1276    3.85    141   1204.0 203.05  157      216    10.7     3
## 217  608    3.31     65   1790.0 151.90  210      298    10.8     4
## 219  215    4.17     67    936.0 134.85   85      176     9.6     3
## 220  426    3.22     96   2716.0 210.80  113      228    10.6     2
## 221  360    3.65     72   3186.0  94.55  154      269     9.7     4
## 222  372    3.38    227   2310.0 167.40  135      240    12.4     3
## 223  448    2.43    123   1833.0 134.00  155      210    11.0     4
## 224  309    3.66     67   1214.0 158.10  101      309     9.7     3
## 225  274    3.66    108   1065.0  88.35  135      251    10.1     2
## 226  223    3.70     39    884.0  75.95  104      231     9.6     3
## 227  316    3.35    172   1601.0 179.80   63      394     9.7     2
## 228  215    3.35     41    645.0  93.00   74      165     9.6     3
## 229  191    3.05    200   1020.0 175.15  118      139    11.4     4
## 230  302    3.41     51    310.0  83.70   44       95    11.5     4
## 231  518    1.96    115   2250.0 203.05   90      190    10.7     4
## 232  267    3.02     47   1001.0 133.30   87      265    10.6     3
## 233  514    3.06    412   2622.0 105.40   87      284     9.8     4
## 234  578    3.35     78    976.0 116.25  177      322    11.2     2
## 235 1336    4.16     71   3510.0 209.25  111      338    11.9     3
## 236  253    3.79     67   1006.0 139.50  106      341     9.7     3
## 237  442    2.95    105    820.0  85.25  108      181    10.1     3
## 239  300    2.94    231   1794.0 130.20   99      319    11.2     4
## 240  232    3.72     24    369.0  51.15  139      326    10.1     3
## 241  316    3.62    308   1119.0 114.70  322      282     9.8     4
## 242  354    2.97     86   1553.0 196.85  152      277     9.9     3
## 243  468    2.81    139   2009.0 198.40  139      233    10.0     4
## 244  350    3.22    121   1268.0 272.80  231      270     9.6     3
## 245  273    3.65     48    794.0  52.70  214      305     9.6     3
## 246  387    3.77     63   1613.0 150.35   33      185    10.1     4
## 247 1712    2.83     89   3681.0 158.10  139      297    10.0     3
## 248  324    3.51     39   1237.0  66.65  146      371    10.0     3
## 249  242    3.20     35   1556.0 175.15   71      195    10.6     4
## 250  299    3.36     23   2769.0 220.10   85      303    10.9     4
## 251  227    3.61     40    676.0  83.00  120      249     9.9     2
## 252  246    3.35    116    924.0 113.15   90      317    10.0     4
## 253  243    3.03    380    983.0 158.10  154       97    11.2     4
## 254  227    3.75    121   1136.0 110.00   91      264    10.0     3
## 255  193    3.85     35    466.0  53.00  118      156    10.3     3
## 256  336    3.74     48    823.0  84.00  108      242     9.7     3
## 257  280    4.23     36    377.0  56.00  146      227    10.6     2
## 258  414    3.44     80   1003.0  99.00   55      271     9.6     1
## 259  277    2.97     42   1110.0 125.00  126      221     9.8     3
## 260  232    3.59    188   1120.0  98.00  128      248    10.9     4
## 262  322    3.06     65   2562.0  91.00  209      231     9.5     3
## 263  432    3.57     45   1406.0 190.00   77      248    11.4     4
## 264  356    3.12    188   1911.0  92.00  130      318    11.2     3
## 265  348    3.20    121    938.0 120.00  146      296    10.0     4
## 266  318    3.32     52    613.0  70.00  260      279    10.2     3
## 267  222    2.33    138    620.0 106.00   91      195    12.1     4
## 268  344    2.75     16    834.0  82.00  179      149    11.0     4
## 269  374    3.50    143   1428.0 188.00   44      151    10.1     2
## 270  448    3.74    102   1128.0  71.00  117      228    10.2     3
## 271  321    3.50     94    955.0 111.00  177      289     9.7     3
## 272  226    2.93     22    674.0  58.00   85      153     9.8     1
## 273  328    3.46     75   1677.0  87.00  116      202     9.6     3
## 275  572    3.77     77   2520.0  92.00  114      309     9.5     4
## 276  219    3.85     67    640.0 145.00  108       95    10.7     2
## 277  317    3.56     44   1636.0  84.00  111      394     9.8     3
## 278  338    3.70    130   2139.0 185.00  193      215     9.9     4
## 279  198    3.77     38    911.0  57.00   56      280     9.8     2
## 280  325    3.69     69   2583.0 142.00  140      284     9.6     3
## 281  175    2.10    220    705.0 338.00  229       62    12.9     4
## 282  304    3.52     97   1622.0  71.00  169      255     9.5     4
## 283  412    3.99    103   1293.0  91.00  113      422     9.6     4
## 284  291    3.44     75   1082.0  85.00  195      251     9.5     3
## 285  253    3.48     65    688.0  57.00   80      252    10.0     1
## 286  310    3.36     70   1257.0 122.00  118      143     9.8     3
## 287  373    3.46    155   1768.0 120.00  151      258    10.1     4
## 288  310    3.89    107    637.0 117.00  242      298     9.6     2
## 289  416    3.99    177    960.0  86.00  242      269     9.8     2
## 290  294    3.57     33    722.0  93.00   69      283     9.8     3
## 291  339    3.18    123   3336.0 205.00   84      304     9.9     4
## 292  546    3.73     84   1070.0 127.00  153      291    11.2     3
## 293  194    2.98    196    815.0 163.00   78      122    12.3     4
## 294 1000    3.07     88   3150.0 193.00  133      299    10.9     4
## 295  646    3.83    102    855.0 127.00  194      306    10.3     3
## 296  328    3.31     62   1105.0 137.00   95      293    10.9     4
## 297  275    3.43    100   1142.0  75.00   91      217    11.3     4
## 298  340    3.37     73    289.0  97.00   93      243    10.2     3
## 299  342    3.76     90   1653.0 150.00  127      213    10.8     3
## 301  393    3.57     50   1307.0  74.00  103      295    10.5     4
## 302  335    3.95     43    657.0  52.00  104      268    10.6     2
## 303  372    3.25    108   1190.0 140.00   55      248    10.6     4
## 304  219    3.93     22    663.0  45.00   75      246    10.8     3
## 305  426    3.61     73   5184.0 288.00  144      275    10.6     3
## 306  239    3.45     31   1072.0  55.00   64      227    10.7     2
## 307  273    3.56     52   1282.0 130.00   59      344    10.5     2
## 308  246    3.58     24    797.0  91.00  113      288    10.4     2
## 309  260    2.75     41   1166.0  70.00   82      231    10.8     2
## 310  434    3.35     39   1713.0 171.00  100      234    10.2     2
## 311  247    3.16     69   1050.0 117.00   88      335    10.5     2
## 312  576    3.79    186   2115.0 136.00  149      200    10.8     2
## 
## attr(,"mtry")
## [1] 5
## attr(,"n_obs")
## [1] 276
## attr(,"n_tree")
## [1] 1
## attr(,"names_y")
## [1] "time"   "status"
## attr(,"names_x")
##  [1] "trt"      "age"      "sex"      "ascites"  "hepato"   "spiders" 
##  [7] "edema"    "bili"     "chol"     "albumin"  "copper"   "alk.phos"
## [13] "ast"      "trig"     "platelet" "protime"  "stage"   
## attr(,"names_x_ref")
##  [1] "trt_placebo" "age"         "sex_f"       "ascites_1"   "hepato_1"   
##  [6] "spiders_1"   "edema_0.5"   "edema_1"     "bili"        "chol"       
## [11] "albumin"     "copper"      "alk.phos"    "ast"         "trig"       
## [16] "platelet"    "protime"     "stage"      
## attr(,"types_x")
##  [1] "factor"  "numeric" "factor"  "factor"  "factor"  "factor"  "factor" 
##  [8] "numeric" "integer" "numeric" "integer" "numeric" "numeric" "integer"
## [15] "integer" "numeric" "ordered"
## attr(,"n_events")
## [1] 111
## attr(,"max_time")
## [1] 4556
## attr(,"unit_info")
## list()
## attr(,"fctr_info")
## attr(,"fctr_info")$cols
## [1] "trt"     "sex"     "ascites" "hepato"  "spiders" "edema"   "stage"  
## 
## attr(,"fctr_info")$lvls
## attr(,"fctr_info")$lvls$trt
## [1] "d_penicill_main" "placebo"        
## 
## attr(,"fctr_info")$lvls$sex
## [1] "m" "f"
## 
## attr(,"fctr_info")$lvls$ascites
## [1] "0" "1"
## 
## attr(,"fctr_info")$lvls$hepato
## [1] "0" "1"
## 
## attr(,"fctr_info")$lvls$spiders
## [1] "0" "1"
## 
## attr(,"fctr_info")$lvls$edema
## [1] "0"   "0.5" "1"  
## 
## attr(,"fctr_info")$lvls$stage
## [1] "1" "2" "3" "4"
## 
## 
## attr(,"fctr_info")$keys
## attr(,"fctr_info")$keys$trt
## [1] "trt_d_penicill_main" "trt_placebo"        
## 
## attr(,"fctr_info")$keys$sex
## [1] "sex_m" "sex_f"
## 
## attr(,"fctr_info")$keys$ascites
## [1] "ascites_0" "ascites_1"
## 
## attr(,"fctr_info")$keys$hepato
## [1] "hepato_0" "hepato_1"
## 
## attr(,"fctr_info")$keys$spiders
## [1] "spiders_0" "spiders_1"
## 
## attr(,"fctr_info")$keys$edema
## [1] "edema_0"   "edema_0.5" "edema_1"  
## 
## attr(,"fctr_info")$keys$stage
## NULL
## 
## 
## attr(,"fctr_info")$ordr
## [1] FALSE FALSE FALSE FALSE FALSE FALSE  TRUE
## 
## attr(,"n_leaves_mean")
## [1] 25
## attr(,"n_split")
## [1] 5
## attr(,"leaf_min_events")
## [1] 1
## attr(,"leaf_min_obs")
## [1] 5
## attr(,"split_min_events")
## [1] 5
## attr(,"split_min_obs")
## [1] 10
## attr(,"split_min_stat")
## [1] 3.841459
## attr(,"cph_method")
## [1] "efron"
## attr(,"cph_eps")
## [1] 1e-09
## attr(,"cph_iter_max")
## [1] 1
## attr(,"cph_do_scale")
## [1] TRUE
## attr(,"net_alpha")
## [1] 0.5
## attr(,"net_df_target")
## [1] 5
## attr(,"numeric_bounds")
##          age  bili  chol albumin copper alk.phos      ast   trig platelet
## 10% 35.66324 0.600 212.5  3.0100  25.50   659.00  61.3150  63.50   142.50
## 25% 41.51266 0.800 249.5  3.3100  42.75   922.50  82.4575  85.00   200.00
## 50% 49.70979 1.400 310.0  3.5450  74.00  1277.50 116.6250 108.00   257.00
## 75% 56.58453 3.525 401.0  3.7725 129.25  2068.25 153.4500 151.25   318.25
## 90% 62.88433 7.250 567.0  4.0100 213.50  3788.00 197.6250 195.00   383.50
##     protime
## 10%     9.7
## 25%    10.0
## 50%    10.6
## 75%    11.2
## 90%    12.0
## attr(,"trained")
## [1] TRUE
## attr(,"n_retry")
## [1] 3
## attr(,"orsf_type")
## [1] "cph"
## attr(,"f_beta")
## function(x) x
## <bytecode: 0x0000000019536b70>
## <environment: 0x0000000022999498>
## attr(,"f_oobag_eval")
## function(x) x
## <bytecode: 0x0000000019533388>
## <environment: 0x0000000022999498>
## attr(,"type_oobag_eval")
## [1] "H"
## attr(,"oobag_pred")
## [1] TRUE
## attr(,"oobag_eval_every")
## [1] 1
## attr(,"importance")
## [1] "anova"
## attr(,"weights_user")
## numeric(0)
## attr(,"tree_seeds")
## integer(0)
```

---

## Your guide to a friendly modeling package

[click here](https://tidymodels.github.io/model-implementation-principles/) to view this book from `tidymodels` online

<iframe src="https://tidymodels.github.io/model-implementation-principles/index.html" width="100%" height="400px" data-external="1"></iframe>

---

## Thank you!

Incredible team members:

- `aorsf`: Sawyer Welden, Kristin Lenoir, Jaime L. Speiser, Matthew W. Segar, Ambarish Pandey, and Nicholas M. Pajewski

- today's talk: Jaime Lynn Speiser, Joseph Rigdon, Heather Marie Shappell, Nathaniel Sean O'Connell, and Michael Kattan.

--

Research reported in this presentation was supported<sup>1</sup> by

- Center for Biomedical Informatics, Wake Forest University School of Medicine.

- National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, through Grant Award Number UL1TR001420.

.footnote[
<sup>1</sup> The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
]