in this video I will demonstrate how to perform a multivariate analysis of variance also known as a manova this technique is an extension of analysis of variance and we use it when we have more than one dependent or outcome variable now these dependent variables typically are assumed to be related in some way or there should be at least some conceptual reason for considering them together or measuring them together so manova compares the groups and tells you whether the mean differences between the groups on the combination of the dependent variables is likely to have curd by chance or is it due to the fact of the treatment or the group designation so in order to do this man OVA creates a new summary or aggregate dependent variable which is basically a linear combination of each of the independent or each of the individual dependent variables so it then performs an analysis of variance using this new combined or aggregate dependent variable and so the manova will be able to tell you if there’s a significant difference between your groups on this composite dependent variable and then if there is you can then look at whether or not the treatment has an effect on the individual dependent variables one at a time so some people might say well why not just do a series of innova separately for each outcome or dependent variable and many researchers will do that unfortunately by doing a whole series of analyses on the same grouping of data you run that risk of an inflated type one error and this is similar to what we run into with doing several t tasks as opposed to doing one way ANOVA so the more analysis you do the more likely you are to find a significant result even if in a reality there’s no difference between your groups so we could have a greater type or inflation of type one error so the advantage that of using manova is that it controls or adjust for this increased risk of a type 1 error but the cost of that is it’s a much more complex set of procedures and it has several assumptions additional options that need to be met now if you decide well manova sounds too complicated for me I don’t really want to do that I’d rather do a series of individual Nova’s you can still do that but what we’ll need to do is in order to reduce the risk of a type 1 error you’ll need to set a more stringent alpha value and one of the ways to do that is to perform the bonferroni adjustment which is where we take our normal alpha level which you’re going to test the hypothesis at in this case maybe 0.05 and then divide it by the number of tests that you intend to perform so if you have three dependent variables you would divide 0.05 by three which works out to be point zero one seven after we round up and then we would use this new value as your cutoff in order to make your hypothesis decision so differences between your groups would need a probability value or p-value of less than 0.01 seven before you could consider them statistically significant so manova can be used in a one-way situation which is what we’re going to do in our example here you can do factorial or two-way as well as higher-order factorial designs like Man O or man Cova or factorial man coma so in the example that we’re going to do here today we’re going to have one independent variable with three levels and we’re going to have two outcome variables or two dependent variables so we’re considered to be a one way manova with two outcomes and so what we’re going to do in this in this example is we’re going to compare the effect of three different types of surgical techniques on shoulder injuries so patients would be randomly assigned to one of three surgical techniques in order to repair a shoulder injury and the two outcomes that we’re going to measure our recovery time which is the number of weeks it would take them to return to being able to perform a list of kind of everyday functional activities so however many weeks it takes them to be able to do that list of activities without pain we’re calling recovery time and then we’re also using what’s known as like rate of change or grr OC scale and this is a subjective measurement that patience will give us to tell us how much improvement they may have received after the intervention in this case the surgery so this gr OC scale ranges from minus 7/2 plus 7 and so a plus 7 score would mean they had very significant improvement in their condition a minus 7 score would indicate that they had very significant worsening of their condition

a score of zero would mean there’s really no change there no better often they’re no worse off so those are our two outcomes we’re gonna measure what a fact the technique has on recovery time presumably certain techniques will have a faster recovery time than others and then we’re also measuring this global rate of change presumably certain techniques will have a greater level of improvement versus worsening and so we’re going to compare those two things and so we have one in this case categorical independent variable with three levels we have three different surgical techniques we have to in this case or we could have more continuous or quantitative outcome variables that we are going to presume are somewhat related to one another so it’s by intuition we would assume that recovery time and improvement or lack of improvement will be related to another one another we’re going to test that so we’ve got several assumptions that we need to tasked with manova and some of these are going to sound familiar from doing ANOVA’s but there are some new additional assumptions that we need to look at as well one of the assumptions the first assumption is sample size and I’ll talk about these a little more detail as we go through our example but we have sample size we need to test the normality of the outcome variables we also need to make sure we don’t have any outliers present we need to make sure our variables are linearly related we also need to make sure there is not any multicollinearity when I’ll explain that in a little bit and we also need to make sure we have homogeneity of variance as well as homogeneity of covariance okay so let’s look at the the assumptions first sample size is the first one and here we typically use a rule of thumb if we need to have more cases in each group or more subjects in each group then we have dependent variables so ideally we’ll have many more than this but this is kind of the absolute minimum so having a larger sample can help us kind of get away with violations of some of the other assumptions like normality or some of the other things we can look at so the minimum required number of cases in each in each group in this example would be 2 because we have two dependent or outcome variables now we have a total of 6 groups we have three levels of our independent variable surgery surgery technique number one two and three and then we have two dependent variables so we have a total of six groups here so the number of cases that we’re gonna see in each group is going to be provided as part of the manova output and then we’ll be able to examine whether or not we’ve met that assumption now the next is normality and so even though the significance tasks of Menno of our based on the multivariate normal distribution so in practice that’s it’s pretty robust and it’s pretty it’s able to deal with modest violations of normality except where the violations might be due to outliers so when we have a sample size of at least 20 in each group we kind of assume that’s going to be a robust enough that we can overcome any differences or any limitations of normality so we still need to check univariate normality of each outcome which we do through the explore function and looking at the skewness score and then we’re also going to check multivariate normality and I’ll show you how to do that a little bit later and so these procedures will also help us determine if we have any any outliers and we can use the outlier labeling technique as I discuss in one of the other videos to help us identify we have any potential outliers now talking more about outliers manova is very sensitive to outliers so if we’ve got data points or scores that are much different than the rest of the scores so we need to check for univariate outliers again using outlier labeling technique for each of our dependent variable separately and then we also are going to look for multivariate outliers so multivariate outliers are participants with a kind of strange unusual combination of scores on the various dependent variables so in other words they might have a very high score on one variable but a very low score on another variable so again we’re going to check for univariate outliers by using the explore function and then using the outlier labeling technique now in order to check for multivariate outliers I’m going to kind of walk you through the procedure to do that so to test for these four multivariate normality or outliers we’re going to use something called the Mahalanobis distance and spss

is going to help us calculate that and we’re going to be using a regression function to do that so this Mahalanobis distance is the distance of a particular case or score from the center of the centroid of the remaining cases where the centroid is basically the point created by all the means of all the variables so it’s kind of a center score of all created by all the scores so this analysis will help us pick up on any cases that have a strange pattern of scores across all or across both of the dependent variables so what’s going to happen is this procedure will create a new variable in my our data file and it’s going to be labeled NH underscore 1 and you’ll see that when we run the analysis so each person or subject receives a value on this variable that indicates the level or the degree to which their pattern of scores differ from the remainder of the sample so to decide whether a case is an outlier we need to compare this this distance value against a critical value and so we use a chi-square critical value table to do that and what I’ll do is in the description of this video I’ll post the table that you can use as a reference to help determine if you’ve got distances that are outside of the acceptable level so if an individual’s mah underscore one score exceeds this critical value it’s considered an outlier and so manova can tolerate a few outliers a handful of outliers particularly if their scores are not too extreme or not too far away from this critical value and if we have a reasonable sized data file in other words a reasonable sample size so if we’ve got close to you know 20 subjects in each cell and we’ve got a few outliers we’re probably going to need to delete those but if we’ve got 50 or 75 or a hundred subjects per cell we can easily tolerate a handful of outliers so if we have too many outliers or very extreme scores in other words a Mahalanobis score that is wait quite a bit above the critical value we’re probably going to need to consider deleting those cases or maybe transforming the variables that are involved so let’s go ahead and let me show you how to run this this procedure detect test for a multivariate normality so the first thing we want to do is go to the analyze menu we want to select regression and then linear so we want to enter the variables our first enter in the variable identifies each one of our cases which in this case the surgical technique that’s re our categorical or independent variable and we put that in the dependent box which seems counterintuitive but that’s how we run this analysis and then in the independence box we put our outcome variables in this case weeks until return to normal and then that global rating of change score and we click on the Save button and then we want to make sure we’ve checked under the distances box we check that mahalo Tobias distance option so you can see I’ve checked that hey once we’ve done that we click on the ok buy or click on the continue button excuse me and then we click on ok now if we go back to our data file you’ll see that we have created this new variable mah underscore one okay so that’s the variable that we’re going to use to help us determine if we have any outliers now if we go back to our output file ok the box that we are interested in their table we’re interested in is the very last one let’s label the residual statistics and what we want to look for is the maximum value listed under the Mahalanobis distance row so here we see here the MA a little B is distance and our maximum score is eleven point two one eight now we’re going to take note of that value in this case it’s eleven point two which doesn’t seem like a very high value so we’re going to compare this number to that critical value I mentioned earlier and so this critical value is again determined by using a chi-square table with the number of outcome or dependent variables we have serving as our degrees of freedom value and then the Alpha value our p-value we’ll use to determine our critical value is point zero zero one so again I put a table and then in the descriptor of for this video description section of this video and you’ll be able to see this table and I’ll include four studies up two up to ten outcomes ten dependent

variables so in this case we have two dependent variables so our critical value for determining if we have an outlier it’s going to be thirteen point eight two so if we compare that to our max score of eleven point two one eight we can see our max score does not exceed that critical value of thirteen point eight two so that means we do not have any multivariate outliers if this max score did exceed that thirteen a – then I would indicate we do have outliers and so if we wanted to define those outliers and take a look at what their scores really are relative to and how many there are relative to this critical value we would have to sort the cases let me show you how to do that just for demonstration purposes so in our example we did not exceed the critical value so we would determine that we met the assumption of no multivariate outliers but if our critic if this value did exceed the critical value we need to find those values and how many there are so go back to our data file and we can see here we’ve got our our value so what we want to do is sort this variable by the value okay so we’re gonna go to data and then sort cases and what it’ll do is it’ll rearrange the data so we have the highest mah one scores at the top and we can see what those scores are and how many there are so we move our new Val variable that we created this model it’ll be its distance and move that into sort by we want to choose a sort order of descending so I’ll list the highest scores first so again we go back to our data file now and we can see we’ve got our highest scores listed at the top and then descending to the lower scores so here’s where we’d be able to examine what those actual Mahalanobis distance scores are so we’d be able to identify number one how large that score is relative to the critical value and then how many subjects might have scores that exceed that critical value so again in this example we don’t have any but we this is where we’d be able to see this so for example if one of the subjects had a mile it’ll be its distance score that was maybe one or two times larger than our critical values and now that would be a large difference and I probably want you can you know figure out how I can either transform the variable maybe delete that individual if I maybe only had one or two individuals that that critical value I probably leave them if I had you know more than about two percent of my total subjects exceeding that critical value that I would probably have to consider removing those subjects or transforming the variable so that’s how I would deal with determining if I had these multivariate outliers but again in this case we don’t have any so we’ve we’ve met that assumption okay the next assumption we have to examine is the assumption of linearity and so this refers to the presence of a straight line or linear relationship between each pair in this case only one pair of dependent or outcome variables so if we have multiple more than two outcome variables we’d have multiple pairs of variables that we’d have to determine if they’re linear linearly related so we can assess this in a number of ways and probably the most straightforward of this buts but subject somewhat subjective is to generate a matrix of scatter plots between each pair of variables again in this case only one pair and then separate it out by our groups in this case type of surgery so how we do this we go to the graphing function select the legacy dialogues and then scatter slash dot so we’re going to determine a scatter plot here so we click on the matrix scatter and then click define and then all of our outcome variables we move into the box labeled matrix variables so our recovery time in weeks and then our global rating of change going to the matrix variables box and then in the rows box goes our independent variable in this case surgical technique okay the next thing we want to do is click on the options button and make sure that we have a list of exclude cases variable by by variable in our example here we don’t have any missing cases but if you did have missing cases you would then have them be excluded variable by variable so that those won’t be included in any analysis all right then we click on the continue button and then we click on OK

and so what we’re gonna examine down here at the bottom are these scatter plots and so what we’re looking for is to see if there’s a general linear relationship here and so if we were to kind of draw an imaginary line of best fit on each of these we can see that these generally trend from lower left to upper right now some of these look almost more like squares than bubbles that trend so that’s a little questionable that we have the true linear relationship what we’re really looking for are maybe a trend that might have more of a curvilinear relationship down books like upside down U or maybe right-side up you but in this case we have a generally linear relationship but it’s a little suspect because we have especially here we have kind of a sprat of scores it almost makes this more look more like a square than it does like an oblong object now this looks a little more linear spread out but it’s a little more linear this doesn’t quite look as linear so you could argue that we’ve met the assumption but we’re going to be a little conservative here and say it doesn’t look like we’ve met that assumption that’s okay we don’t need to stop but we are going to change how we perform or how at least we assess the outcomes of our actual manova and I’ll talk about that when we get to that point but this is how we assess linearity of our outcome variables now the next assumption we have to determine is looking at multicollinearity and manova works best when the dependent variables are moderately correlated we don’t want them to have weak or low correlations we don’t want them to be too highly correlated so for example if we have low correlations we should probably consider running separate univariate analysis of variance for each of our outcome variables if the dependent variables are too highly correlated this is referred to as multicollinearity now this can occur when when one of your variables is a combination of other variables so for example the total scores of a measurement scale that is made up of certain sub scales that are also being examined as outcome variables also if we’ve got two variables that are redundant in other words they’re measuring pretty much the same construct we can have this this very high cold multicollinearity so what we want to do is assess whether or not we have this either too low or too high of a relationship what we want it’s kind of that middle range of a relationship between these variables and so how we’re going to test this is we’re going to basically run a bivariate correlation so we want to go to the analyze menu click correlate and then bivariate and we want to move our to outcome variables into the variables box we want to examine the relationship between these two variables and then we click the ok button and so what we want to examine here is this R value so we can see that our value between this gives us an idea of the strength of the relationship between these two variables now our values can range between minus 1 and plus 1 so the closer the R value is to 1 the stronger the relationship between these two values whether it’s negative or positive so what we like to see what’s optimal is having an R value that ranges somewhere between 0.2 and around 0.8 if it’s less than 0.2 these two variables aren’t related enough if it’s greater than 0.8 or 0.9 depending on on who you you read that’s too much of a relationship so we like the value to be kind of in that middle range of 0.2 to 0.8 so in this case our value of our two variables is 0.34 9 which is in the middle there just as close to the bottom end but it sits in that middle range so these two variables are related to one another but not two related so we see appears we’ve met that assumption then of not having multicollinearity or of not having too little of a relationship which is we referred to as again multi : or not having enough relationship so our last assumption we need to determine then is this homogeneity of variance and covariance matrices so this is generated as part of our actual manual analysis and the test we use to assess this is known as boxes M test and so we’re gonna look at that when we look at our Manoa bi output to determine that last

assumption so up to this point we’ve met the majority of our assumptions or when we’re a little skeptical of is that linearity assumption and so how we examine our manova output will be a little bit different I’ll explain to that when we get to that point so now our next step is to go ahead and actually perform the manova and so we’re going to again perform a one way manova using surgical technique as our independent variable and we’ve got two outcome variables weeks of recovery time and then global rating of change so how we perform the actual manova we’re gonna go back to the analyze menu we’re gonna choose general linear model and then we’re going to click on the multivariate option and we reason which clicking our multivariate is because we have multiple outcome variables we only have one independent variable but we have multiple outcome variables okay so the first thing we want to do is move our dependent variables into the dependent variables box so in this case that’s weeks and of recovery time and then global rating of change let’s go into our dependent variables box in the fixed factors box goes our independent variable in this case surgical technique and then we want to click on the model button we want to make sure that full factorial is chosen we want to make sure that the sum of squares is listed as Type three and those are the defaults so those are typically will be in place we want to double check that that is the case all right then we click on continue let me click on our options button and so we want to highlight surgical technique and move that over into the right hand box so we can see the means for each of our groups and then in the display box down here we want to click on descriptive statistics we want to see effect size we want to see observe power and we also want to see homogeneity test and that’s what we’re going to get our boxes test result all right then we click on continue now the last button we need to do is address our post hoc test now if we have a significant difference for our aggregate overall variable we’re then going to examine whether or not the individual outcome variables are also showing significance if they are then we’ll need to do a post hoc on each of those individual outcome variables so again we’ve got three levels of our treatment so let’s say for example we find significance for our aggregate variable and then we also find significance for global rating of change we then are going to want to do a post hoc to see which individual group has significant differences compared to the others on this global rating of change so we’re gonna choose post hoc test know we don’t know if we’re going to need to use them or not yet but we’re going to go ahead and make that choice now so that will have it available to us if we do need it okay so we want to highlight surgical technique our independent variable and move it into the right-hand box and then because we have it’s a one-shot measurement weeks of recovery and then global rating of change we don’t need to use the bonferroni post hoc but we have unequal group sizes so we want to choose the Schiff a post hoc if we had equal group sizes we could choose the two key tasks but we do not so we’re going to choose the shift PHA test so we then will click continue and then we can click OK so the first thing we’ll see in our output then is going to be our descriptive so here are the mean scores for each of our groups so weeks of return to function here’s the mean number of weeks for each group our three techniques and then global rating of change again here’s our three means of scores so as we can see here all three groups took roughly the average return to normal was about 14 weeks and we can see that the average global rating of change for all three groups was positive it was a bits above zero so they all showed some improvement but it appears that the standard surgical technique group had a higher improve or greater improvement remember a score of zero would mean there’s no improvement negative or positive where there’s no change negative or positive so everyone had a positive change now one of the first assumptions we can look at is that sample size assumption that we talked about in each group we took the number of independent variable levels that we had in this case 3 and multiplied it by the number of outcomes we had in this case 2 so that would give us a situation when we want to have at least 6 subjects in each cell and we obviously have many more than that and we also have more than that at kind of twenty number that we think

of as being another minimum rule of thumb so we’ve got way more than that we’ve got good sample sizes and matter of fact our first two groups have very large sample sizes over 250 but our third group still has a decent sample size close to 100 so we seem to have met that assumption as well as having adequate sample sizes okay the next thing we want to look at is boxes tasks to determine if we have met that assumption of homogeneity of variance and what we do is we look at the SI g value which is the bottom value in the box and we want that to exceed point zero zero one okay if we feel or if we see that we’ve exceeded point zero zero one then we have not violated that assumption of homogeneity so in this case our our significance value is point nine six one so that’s that’s way above point zero zero one so we know that we have equality of variance for the aggregate outcomes now we’re gonna test the quality of variance for the individual dependent variables and in a few minutes but we’ve met the assumption for boxes tests so that we’re in good shape there now the next thing we want to do is examine whether or not the independent variable had an effect on the aggregate outcome so again we took those two dependent variables and we have kind of an aggregated outcome that we want to see if the treatment had an effect on it okay and so when we look at the multivariate test table this is what’s going to tell us whether or not the treatment had an overall effect on the variables the outcome variables so we want to find the part of the table that’s labeled surgical technique and the first two rows you’ll see here are labeled p’lice trace and Wilks lambda now we can use either one of those F scores to determine whether or not we’ve met the null hypothesis of no effect of our of our independent variable now if we’ve met all of our assumptions that we’ve talked about up to this point we would use the Wilks lambda to make a hypothesis decision if we have violated an assumption in this case we talked about we may have violated the linearity assumption we would then use p’lice trace so for example if we violated normality if we violated the linearity if we violated box’s test and we violated any of those assumptions we’d want to use peláez trace to make our initial what we call our omnibus f test to determine if the aggregate outcome variable has been affected by the treatment so in this case because we were concerned about that linearity assumption we’re going to use peláez trace to make that decision so we follow that over when we look at our F score in this case it’s five point two oh eight and then we find the significance value associated with that that is less than 0.05 so we can make the assumption or make the conclusion excuse me that the treatment surgical technique has an effect and the aggregate of our outcomes so we’ve kind of met our first hypothesis decision or made our first hypothesis ition we’re going to reject the null hypothesis and say that the treatment did have an effect on the aggregate outcome so we can move on and do more additional analyses so we can also look at the partial at a squared which is the effect size and if we multiply this by a hundred that will give us a percentage and what this percentage will tell us is what percentage of the variance in the outcome variables is explained by our treatment so in this case we’ve got a fairly small percentage of outcomes or a variance that’s explained by our treat so only about one and a half percent of the variance in our aggregate outcome is explained by our treatment so then that’s not a lot of a fact so that’s a pretty weak effect size we can also look at the observed power we have very adequate power again if we have a point 8’o value or greater that indicates that we have adequate power and we’re likely to see the same result the vast majority of the time so again if we were to do this 100 times about 97 percent of the time we would see the same statistical significance so that’s a good thing as well so now that we we’ve looked at the first null hypothesis and we were able to reject that null and say that the treatment did have an effect on the overall aggregate variables we can move to the next level and now we can start to look at the effect of the of the intervention on the individual outcomes in other words weeks of recovery and then global rating of change so the next thing we can examine then is Levine’s test to see if the individual outcomes have equality of variance and we’re going to use in this case a significance

value of 0.05 and so both of these significance values are greater than 0.05 so we’ve met the assumption of equality of variance for both of our individual outcomes so that’s also a good new good news so we’ve met that assumption now the next thing we want to do in the next table we can look at is label test of between subjects of facts and this is going to tell us the effect of the treatment on the individual outcomes one at a time so we’re going to look at the effect of surgical technique on weeks of recovery by itself and then surgical technique its effect on global rating of change now when we do that we’re gonna have to adjust our p-value that we’re going to test for statistical significance so we’re going to use the bonferroni correction in this case so we’re going to take our original p-value of 0.05 and divide it by the number of outcomes and we have in this case two so that gives us a new significance level of 0.025 that we’re then going to use to test for significance to see if the TEC surgical technique had an effect of our individual outcomes so what we want to do that is go to the section of this table labeled surgical technique and you’ll see our two outcomes have separate rows so the first thing we can look at is weeks until return the minimal function so it’s our recovery time we follow that line over to our p value of 0.09 oh now this does not this is not less than 0.05 so we’d be able to accept the null hypothesis at this at this point and say that surgical technique did not affect recovery time if this value is less than 0.05 we’d be able to say it did affect recovery time there are significant differences somewhere but in this at this point we have to accept the null and say that recovery time is not affected by surgical technique at a statistically significant level so we can accept that null and we can stop with that particular dependent variable we don’t need to move on and do any additional analysis now for global rating of change to want to write below it we do the same thing we follow that over to the to the significance level and this value is less than 0.05 and so we’d be able to make the conclusion that surgical technique did have a significant effect on global rating of change and there are significant differences somewhere among our groups so now we’re going to be able to move on to doing postdocs to figure out which groups are different from each other on global rating of change so we’re able to reject our initial null hypothesis on that aggregate outcome then we looked at the individual outcomes and for recovery time we accepted the null saying that the treatment had no effect recovery time but for global rating of change we’re able to reject the null and say that treatment did have an effect on the rating of change outcome so now we can move on to do post tox for global rating of change to see which techniques were different from each other so now we want to move down oops move down and find the post hoc results which are listed here as multiple comparisons and so again we’re just interested in global radium change that’s the only outcome that had significance and so first we’re gonna compare the standard surgical technique to technique number one new technique number one and we can see the significance value shows that there is significant differences between these two techniques now we look at standard technique compared to new technique number two and we can see that there is no significance this value is greater than that point oh two five value that we’re using as our our new significance value so we would say that there is no difference between standard technique and technique number two now we can also look at comparing technique number one to technique number two and we can see again the significance value is greater than that point oh two five value so the only significant differences we can find among techniques are between standard technique and technique number one now we can look up just a little bit here and we can look at the means of these groups and so if we look at the mean for a standard group they had a mean global rating of change of one point six seven and technique number one group had a mean change of only point five six six so those two are statistically significant from each other in their differences technique number two and standard technique are different numerically but they are not statistically significant in their differences so again in choosing which technique would be most effective it appears that the standard surgical technique is most effective when it

comes to global rating of change especially compared to technique number one but does not appear to be significantly different from technique number two when it comes to weeks of return or weeks recovery time in weeks there are no significant differences among the three options now one more thing we can look at to see if this difference between standard technique and technique number one is a consistent difference in other words we’ll see the same sort of result in multiple samples we can look at the 95% confidence interval so if we look at the lower bound of standard group and the upper bound of technique number one we can see those two boundaries do not overlap in other words 0.192 is different than point than one point three if this technique technique number one had an upper bound that was greater than one point three then that would indicate the boundaries of these two confidence intervals overlap and there could be other samples that would have a very different result in other words there could be a situation where a new technique group number one had a global rating of change that could have been one point three and the standard group could have had a global rating a change of one point three and there’d be no difference between the two groups so that would indicate that there are other groups out in the population that might have a very different result what we’re seeing in our example here is there is a separation between these two confidence intervals meaning that in ninety five percent of all the other samples you could collect from this population there will always be a difference or there will be a difference between the two groups standard group versus new technique number one so that lends some some credibility to the two the difference to saying that there there will be a consistent difference between these two groups in just about every sample you can collect from the population so meaning that there’s a consistent difference among these two treatments that will typically be there ninety-five percent of the time so that gives some idea of a magnitude of the difference in other words it will be very consistent so in summary we demonstrated the the manova technique a one way manova in this case and so we’re able to examine the effect of a treatment on to outcome variables and we also looked at this too we looked at the statistical significance the effect of the treatments on that looked also at the the clinical significance really–it relative to effect size relative to power and also relative to looking at the confidence intervals so hopefully you learned something this technique and good luck using this technique in your own research