American Oil and Gas Reporter - April 2016 - 76
SpecialReport: Drilling Technology
Quality Of Model Fit
Many measures exist to evaluate the
quality of a model fit, including average
absolute error (AAE) and mean squared
error (MSE). These two metrics are
similar, and both attempt to capture the
overall closeness of predictions to the
evaluation data.
AAE is defined as the average magnitude of the difference between the true
response and the predicted response (i.e.,
the average size of the residuals). MSE
measures the average squared difference
between truth and prediction, rather than
the absolute value. AAE has units matching
those of the response, while MSE is
measured in squared units of the response.
Values closer to zero are desirable, since
they indicate smaller deviations between
the truth and predictions (i.e., more accurate prediction).
Figure 2 summarizes the model fitting
results on the Wolfcamp dataset. Each
plot shows the true response (M12CO)
on the horizontal axis and the predicted
response on the vertical axis. Points on
the diagonal dotted line indicate perfect
predictions. Each row of plots shows predictions from one type of model (OLS,
RF, GBM, SVR and KM), while each
column shows results for different model
evaluation types.
The left column shows independent
validation results, where a random 20
percent subset of the wells was held out.
The model was then fit to the remaining
80 percent of the dataset and evaluated
using the held-out dataset. The points in
the plots in the left column correspond
to only those predictions on the held-out
data.
For k-fold cross-validation predictions
(center column), a 10-fold cross-validation
was used as a further refinement of the
fivefold cross-validation approach. The
points in these plots show the actual
FIGURE 3
Decision Tree Separating Top 25 Percent
And Bottom 25 Percent of Producing Wells
PROP< 1,406e+06
LATLEN< 2,756
Bottom 25%
26/0
TVDSS>=-8,100
TVDSS>=-8,294
Bottom 25%
8/2
LATLEN>=5,362
Bottom 25%
7/3
Bottom 25%
21/2
Top 25%
6/15
76 THE AMERICAN OIL & GAS REPORTER
Top 25%
12/58
versus the cross-validated predictions of
each well. The right column shows the
results from training and predicting using
the full dataset.
The independent validation and crossvalidation results tell a much different
story than the full training set predictions.
Compared with OLS regression, many
of the models show a dramatic reduction
in error in both AAE and MSE metrics.
However, this reduction is more modest
for the other methods of model evaluation.
The extreme case is the kriging model,
which is a perfect interpolator, and therefore, forces the model fit through the
training observations by design.
Predictions with the full training set
yield an apparent perfect predictive ability,
which clearly is not going to hold up in
future datasets. However, for a case such
as random forest, it is not so clear that
predictions on the training data are biased.
It is only by comparing with independent
validation and cross-validation plots that
overfitting is revealed.
Basing expectations solely on the full
training-set-based predictions could lead
to an optimistic perception of model accuracy for future data, and disappointment
when those levels of accuracy are not
met. Accordingly, it may be wiser to
adopt a more robust method of model
evaluation, such as a k-fold cross-validation. The results should align better with
actual model performance on future data
collections.
Decision Tree Analysis
One strategy for tackling predictive
problems is to simplify the question being
asked. Rather than using predictive regression models to pinpoint the exact cumulative first-year production for a given
set of well characteristics, suppose the
aim of the exercise is to give a simple
"go/no go" solution on well decisions.
In this case, accurate first-year production prediction is not necessarily required. Instead, the goal is to predict
whether a well will be "good" or "bad"
(relatively high or low M12CO). This
problem can be simplified by changing
the modeling effort from regression to
classification. That is, the response can
be binned into categories, and models
can be used to predict which category a
well falls into.
Decision trees are useful for building
simple, interpretive models to describe
how these categories relate to the predictor
values. The common approach to con-
American Oil and Gas Reporter - April 2016
Table of Contents for the Digital Edition of American Oil and Gas Reporter - April 2016
Contents
American Oil and Gas Reporter - April 2016 - Cover1
American Oil and Gas Reporter - April 2016 - Cover2
American Oil and Gas Reporter - April 2016 - 3
American Oil and Gas Reporter - April 2016 - Contents
American Oil and Gas Reporter - April 2016 - 5
American Oil and Gas Reporter - April 2016 - 6
American Oil and Gas Reporter - April 2016 - 7
American Oil and Gas Reporter - April 2016 - 8
American Oil and Gas Reporter - April 2016 - 9
American Oil and Gas Reporter - April 2016 - 10
American Oil and Gas Reporter - April 2016 - 11
American Oil and Gas Reporter - April 2016 - 12
American Oil and Gas Reporter - April 2016 - 13
American Oil and Gas Reporter - April 2016 - 14
American Oil and Gas Reporter - April 2016 - 15
American Oil and Gas Reporter - April 2016 - 16
American Oil and Gas Reporter - April 2016 - 17
American Oil and Gas Reporter - April 2016 - 18
American Oil and Gas Reporter - April 2016 - 19
American Oil and Gas Reporter - April 2016 - 20
American Oil and Gas Reporter - April 2016 - 21
American Oil and Gas Reporter - April 2016 - 22
American Oil and Gas Reporter - April 2016 - 23
American Oil and Gas Reporter - April 2016 - 24
American Oil and Gas Reporter - April 2016 - 25
American Oil and Gas Reporter - April 2016 - 26
American Oil and Gas Reporter - April 2016 - 27
American Oil and Gas Reporter - April 2016 - 28
American Oil and Gas Reporter - April 2016 - 29
American Oil and Gas Reporter - April 2016 - 30
American Oil and Gas Reporter - April 2016 - 31
American Oil and Gas Reporter - April 2016 - 32
American Oil and Gas Reporter - April 2016 - 33
American Oil and Gas Reporter - April 2016 - 34
American Oil and Gas Reporter - April 2016 - 35
American Oil and Gas Reporter - April 2016 - 36
American Oil and Gas Reporter - April 2016 - 37
American Oil and Gas Reporter - April 2016 - 38
American Oil and Gas Reporter - April 2016 - 39
American Oil and Gas Reporter - April 2016 - 40
American Oil and Gas Reporter - April 2016 - 41
American Oil and Gas Reporter - April 2016 - 42
American Oil and Gas Reporter - April 2016 - 43
American Oil and Gas Reporter - April 2016 - 44
American Oil and Gas Reporter - April 2016 - 45
American Oil and Gas Reporter - April 2016 - 46
American Oil and Gas Reporter - April 2016 - 47
American Oil and Gas Reporter - April 2016 - 48
American Oil and Gas Reporter - April 2016 - 49
American Oil and Gas Reporter - April 2016 - 50
American Oil and Gas Reporter - April 2016 - 51
American Oil and Gas Reporter - April 2016 - 52
American Oil and Gas Reporter - April 2016 - 53
American Oil and Gas Reporter - April 2016 - 54
American Oil and Gas Reporter - April 2016 - 55
American Oil and Gas Reporter - April 2016 - 56
American Oil and Gas Reporter - April 2016 - 57
American Oil and Gas Reporter - April 2016 - 58
American Oil and Gas Reporter - April 2016 - 59
American Oil and Gas Reporter - April 2016 - 60
American Oil and Gas Reporter - April 2016 - 61
American Oil and Gas Reporter - April 2016 - 62
American Oil and Gas Reporter - April 2016 - 63
American Oil and Gas Reporter - April 2016 - 64
American Oil and Gas Reporter - April 2016 - 65
American Oil and Gas Reporter - April 2016 - 66
American Oil and Gas Reporter - April 2016 - 67
American Oil and Gas Reporter - April 2016 - 68
American Oil and Gas Reporter - April 2016 - 69
American Oil and Gas Reporter - April 2016 - 70
American Oil and Gas Reporter - April 2016 - 71
American Oil and Gas Reporter - April 2016 - 72
American Oil and Gas Reporter - April 2016 - 73
American Oil and Gas Reporter - April 2016 - 74
American Oil and Gas Reporter - April 2016 - 75
American Oil and Gas Reporter - April 2016 - 76
American Oil and Gas Reporter - April 2016 - 77
American Oil and Gas Reporter - April 2016 - 78
American Oil and Gas Reporter - April 2016 - 79
American Oil and Gas Reporter - April 2016 - 80
American Oil and Gas Reporter - April 2016 - 81
American Oil and Gas Reporter - April 2016 - 82
American Oil and Gas Reporter - April 2016 - 83
American Oil and Gas Reporter - April 2016 - 84
American Oil and Gas Reporter - April 2016 - 85
American Oil and Gas Reporter - April 2016 - 86
American Oil and Gas Reporter - April 2016 - 87
American Oil and Gas Reporter - April 2016 - 88
American Oil and Gas Reporter - April 2016 - 89
American Oil and Gas Reporter - April 2016 - 90
American Oil and Gas Reporter - April 2016 - 91
American Oil and Gas Reporter - April 2016 - 92
American Oil and Gas Reporter - April 2016 - 93
American Oil and Gas Reporter - April 2016 - 94
American Oil and Gas Reporter - April 2016 - 95
American Oil and Gas Reporter - April 2016 - 96
American Oil and Gas Reporter - April 2016 - 97
American Oil and Gas Reporter - April 2016 - 98
American Oil and Gas Reporter - April 2016 - 99
American Oil and Gas Reporter - April 2016 - 100
American Oil and Gas Reporter - April 2016 - 101
American Oil and Gas Reporter - April 2016 - 102
American Oil and Gas Reporter - April 2016 - 103
American Oil and Gas Reporter - April 2016 - 104
American Oil and Gas Reporter - April 2016 - 105
American Oil and Gas Reporter - April 2016 - 106
American Oil and Gas Reporter - April 2016 - 107
American Oil and Gas Reporter - April 2016 - 108
American Oil and Gas Reporter - April 2016 - 109
American Oil and Gas Reporter - April 2016 - 110
American Oil and Gas Reporter - April 2016 - 111
American Oil and Gas Reporter - April 2016 - 112
American Oil and Gas Reporter - April 2016 - 113
American Oil and Gas Reporter - April 2016 - 114
American Oil and Gas Reporter - April 2016 - 115
American Oil and Gas Reporter - April 2016 - 116
American Oil and Gas Reporter - April 2016 - 117
American Oil and Gas Reporter - April 2016 - 118
American Oil and Gas Reporter - April 2016 - 119
American Oil and Gas Reporter - April 2016 - 120
American Oil and Gas Reporter - April 2016 - 121
American Oil and Gas Reporter - April 2016 - 122
American Oil and Gas Reporter - April 2016 - 123
American Oil and Gas Reporter - April 2016 - 124
American Oil and Gas Reporter - April 2016 - 125
American Oil and Gas Reporter - April 2016 - 126
American Oil and Gas Reporter - April 2016 - 127
American Oil and Gas Reporter - April 2016 - 128
American Oil and Gas Reporter - April 2016 - 129
American Oil and Gas Reporter - April 2016 - 130
American Oil and Gas Reporter - April 2016 - Cover3
American Oil and Gas Reporter - April 2016 - Cover4
http://staging.nxtbook.com/nxtbooks/aogr/201702/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201701/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201612/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201611/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201610/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/pbios2016_programguide/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201609/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201608/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201607/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201606/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201605/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201604/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201603/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201602/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201601/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201512/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201511/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201510/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201509/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201508/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201507/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201506/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201505/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201504/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201503/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201502/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201501/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201412/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201411/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201410/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201409/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/pbios2014_programguide/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201408/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201407/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201406/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201405/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201404/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201403/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201402/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201401/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201312/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201311/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201310/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201309/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201308/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201307/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201306/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201305/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201304/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201303/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201302/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201301/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201212/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201211/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201210/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201209/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/2012_pbios/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201208/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201207/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201206/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201205/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201204/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201203/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201202/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201201/stage.php
http://staging.nxtbook.com/nxtbooks/demo/aogr_clone/stage.php
http://staging.nxtbook.com/nxtbooks/aogr/201112/stage.php
https://www.nxtbookmedia.com