hybrid {puniform} | R Documentation |
Function to statistically combine original studies and replications by means of the hybrid methods and fixed-effect meta-analysis as described in van Aert and van Assen (2018).
hybrid( mo1i, mo2i, moi, roi, sdo1i, sdo2i, sdoi, no1i, no2i, noi, toobs, mr1i, mr2i, mri, rri, sdr1i, sdr2i, sdri, nr1i, nr2i, nri, trobs, m1i, m2i, mi, ri, sd1i, sd2i, sdi, n1i, n2i, ni, tobs, yoi, yri, voi, vri, alpha = 0.05, side )
mo1i |
A vector of means in group 1 for two-independent means for the original studies |
mo2i |
A vector of means in group 2 for two-independent means for the original studies |
moi |
A vector of group means for one-sample means for the original studies |
roi |
A vector of raw correlations for the original studies |
sdo1i |
A vector of standard deviations in group 1 for two-independent means for the original studies |
sdo2i |
A vector of standard deviations in group 2 for two-independent means for the original studies |
sdoi |
A vector of standard deviations for one-sample means for original studies |
no1i |
A vector of sample sizes in group 1 for two-independent means for the original studies |
no2i |
A vector of sample sizes in group 2 for two-independent means for the original studies |
noi |
A vector of sample sizes for one-sample means and correlations for original studies |
toobs |
A vector of t-values for one-sample means and two-independent means for the original studies |
mr1i |
A vector of means in group 1 for two-independent means for the replications |
mr2i |
A vector of means in group 2 for two-independent means for the replications |
mri |
A vector of group means for one-sample means for the replications |
rri |
A vector of raw correlations for the replications |
sdr1i |
A vector of standard deviations in group 1 for two-independent means for the replications |
sdr2i |
A vector of standard deviations in group 2 for two-independent means for the replications |
sdri |
A vector of standard deviations for one-sample means for replications |
nr1i |
A vector of sample sizes in group 1 for two-independent means for the replications |
nr2i |
A vector of sample sizes in group 2 for two-independent means for the replications |
nri |
A vector of sample sizes for one-sample means and correlations for replications |
trobs |
A vector of t-values for one-sample means and two-independent means for the replications |
m1i |
A vector of means in group 1 for two-independent means (deprecated, see Details) |
m2i |
A vector of means in group 2 for two-independent means (deprecated, see Details) |
mi |
A vector of group means for one-sample means (deprecated, see Details) |
ri |
A vector of raw correlations (deprecated, see Details) |
sd1i |
A vector of standard deviations in group 1 for two-independent means (deprecated, see Details) |
sd2i |
A vector of standard deviations in group 2 for two-independent means (deprecated, see Details) |
sdi |
A vector of standard deviations for one-sample means (deprecated, see Details) |
n1i |
A vector of sample sizes in group 1 for two-independent means (deprecated, see Details) |
n2i |
A vector of sample sizes in group 2 for two-independent means (deprecated, see Details) |
ni |
A vector of sample sizes for one-sample means and correlations (deprecated, see Details) |
tobs |
A vector of t-values (deprecated, see Details) |
yoi |
A vector of standardized effect sizes of the original studies (see Details) |
yri |
A vector of standardized effect sizes of the replications (see Details) |
voi |
A vector of sampling variances belonging to the standardized effect sizes of the original studies (see Details) |
vri |
A vector of sampling variances belonging to the standardized effect sizes of the replications (see Details) |
alpha |
A numerical value specifying the alpha level as used in the original study (default is 0.05, see Details). |
side |
A character indicating whether the observed effect size of the
original study is in the right-tail of the distribution (i.e., positive) or
in the left-tail of the distribution (i.e., negative) (either |
Three different effect sizes can be used as input for the
hybrid
function: one-sample means, two-independent means, and raw
correlation coefficients. For each effect size measure, data of the original
studies and replications have to be provided separately. For analyzing
one-sample means, either the group means (moi
and mri
), standard
deviations (sdoi
and sdri
), and sample sizes (noi
and
nri
) for the original studies and replications or t-values (toobs
and trobs
) and sample sizes (noi
and nri
) have to be
provided. For analyzing two-independent means, either the group means of group
1 (mo1i
and mr1i
) and group 2 (mo1i
and mr1i
),
standard deviations of group 1 (sdo1i
and sdr1i
) and group 2
(sdo2i
and sdr2i
), and sample sizes of group 1 (no1i
and
nr1i
) and group 2 (no2i
and nr2i
) for the original studies
and replications have to be provided. It is also possible to analyze
two-independent means by providing t-values (toobs
and trobs
) in
combination with sample sizes of group 1 (no1i
and nr1i
) and group
2 (no2i
and nr2i
) for the original studies and replications.
Correlation coefficients can also be analyzed by supplying the function with
raw correlation coefficients (roi
and rri
) and sample sizes
(noi
and nri
) of the original studies and replications. The
side
argument to specify whether the observed effect size of the
original study is in the right-tail of the distribution (i.e., positive) or
in the left-tail of the distribution should also be specified for every effect size
measure.
It is also possible to specify the standardized effect sizes and its sampling
variances directly via the yoi
, yri
, voi
, and vri
arguments. However, extensive knowledge about computing standardized effect
sizes and its sampling variances is required and specifying standardized effect
sizes and sampling variances is not recommended to be used if the p-values in
the primary studies are not computed with a z-test. In case the p-values in the
original studies were computed with, for instance, a t-test, the p-values of a
z-test and t-test do not exactly coincide and studies may be not statistically
significant according to a z-test.
The hybrid methods assume that the original studies are statistically
significant, so original studies that are not statistically signifcant are
discarded from the analysis. Furthermore, it is assumed that two-tailed
hypothesis tests were conducted in the original studies. In case one-tailed
hypothesis tests were conducted in the original studies, the alpha level has
to be multiplied by two. For example, if one-tailed hypothesis tests were
conducted with an alpha level of .05, an alpha of 0.1 has to be entered into
the hybrid
function.
Previous version
The usage of a previous version of the hybrid
function was more restricted.
Users could only apply the method to a single original study and replication.
Before the addition of the extra functionality to also analyze multiple original
studies and replications, data of the original study and replication were
specified in vectors containing two elements with the first element being
the data of the original study and the second one the data of the replication.
In order to maintain backwards compatibility, it is still possible to analyze
data like this by using the arguments m1i, m2i, mi, ri, sd1i, sd2i, sdi,
n1i, n2i, ni, tobs
. However, using the hybrid
function in this way is
now deprecated.
k |
total number of effect sizes |
krep |
number of effect sizes of replications |
est.hy |
effect size estimate of hybrid method |
ci.lb.hy |
lower bound of hybrid method's confidence interval |
ci.ub.hy |
upper bound of hybrid method's confidence interval |
x.hy |
test statistic of hybrid method's test of null-hypothesis of no effect |
pval.hy |
two-tailed p-value of hybrid method's test of null-hypothesis of no effect |
measure |
effect size measure |
est.hyr |
effect size estimate of hybridR method |
ci.lb.hyr |
lower bound of hybridR method's confidence interval |
ci.ub.hyr |
upper bound of hybridR method's confidence interval |
x.hyr |
test statistic of hybridR method's test of null-hypothesis of no effect |
pval.hyr |
two-tailed p-value of hybridR method's test of null-hypothesis of no effect |
pval.o |
two-tailed p-value of original study |
est.hy0 |
effect size estimate of hybrid0 method |
ci.lb.hy0 |
lower bound of hybrid0 method's confidence interval |
ci.ub.hy0 |
upper bound of hybrid0 method's confidence interval |
x.hy0 |
test statistic of hybrid0 method's test of null-hypothesis of no effect |
pval.hy0 |
two-tailed p-value of hybrid0 method's test of null-hypothesis of no effect |
est.fe |
effect size estimate based on traditional fixed-effect meta-analysis |
se.fe |
standard error of effect size estimate based on traditional fixed-effect meta-analysis |
zval.fe |
test statistic of the null-hypothesis of no effect based on traditional fixed-effect meta-analysis |
pval.fe |
two-tailed p-value of the null-hypothesis of no effect based on traditional fixed-effect meta-analysis |
ci.lb.fe |
lower bound of confidence interval based on traditional fixed-effect meta-analysis |
ci.ub.fe |
upper bound of confidence interval based on traditional fixed-effect meta-analysis |
est.repl |
effect size estimate of replication |
se.repl |
standard error of replication's effect size estimate |
ci.lb.repl |
lower bound of replication's confidence interval |
ci.ub.repl |
upper bound of replication's confidence interval |
stat.repl |
test statistic of replication for testing null-hypothesis of no effect |
pval.repl |
two-tailed p-value of replication for testing null-hypothesis of no effect |
Robbie C.M. van Aert R.C.M.vanAert@tilburguniversity.edu
van Aert, R. C. M., & van Assen, M. A. L. M. (2018). Examining reproducibility in psychology: A hybrid method for statistically combining a biased original study and replication. Behavior Research Methods, 50(4): 1515-1539. doi:10.3758/s13428-017-0967-6
### Apply hybrid method to example on page 5 of van Aert and van Assen (2018). pval.o <- 0.03 # p-value original study pval.r <- 0.3 # p-value replication no1i <- no2i <- 40 # Sample size per group in original study nr1i <- nr2i <- 80 # Sample size per group in replication toobs <- qt(pval.o/2, df = no1i+no2i-2, lower.tail = FALSE) # Observed t-values original study trobs <- qt(pval.r/2, df = nr1i+nr2i-2, lower.tail = FALSE) # Observed t-values replication ### Apply hybrid method hybrid(toobs = toobs, trobs = trobs, no1i = no1i, no2i = no2i, nr1i = nr1i, nr2i = nr2i, side = "right") ### Apply hybrid method to two original studies and two replications noi <- nri <- 50 # Sample size original studies and replicaitons sdoi <- sdri <- 1 sei <- sdoi/sqrt(50) # Standard error ### Generate data pso <- c(0.025/3, 0.025/3*2) psr <- c(1/3, 1/3*2) moi <- qnorm(pso, mean = 0, sd = sei, lower.tail = FALSE) mri <- qnorm(psr, mean = 0, sd = sei, lower.tail = FALSE) ### Apply hybrid method hybrid(moi = moi, noi = noi, sdoi = sdoi, mri = mri, nri = nri, sdri = sdri, side = "right")