THE USE OF QUANTITATIVE RESEARCH METHOD AND STATISTICAL DATA ANALYSIS IN DISSERTATION: AN EVALUATION STUDY

Writing a dissertation is one of the requirements every postgraduate student has to accomplish in completing his/her doctorate program; and it is written based on research. This study addresses the problem concerning the pattern of weaknesses in the students’ research, particularly related to the use of research and statistical methods. Its objectives are to explore the weakness pattern of their research method and statistical data processing and analyze various causes of the pattern. The study is focused on the dissertations written by Universitas Pendidikan Indonesia’s School of Postgraduate Studies’ students that used a quantitative approach based on the review undertaken by the school’s commission from academic year 2012/2013 to 2013/2014. The study employed a qualitative approach and data analysis was done by using logical analysis of the review results. Findings of the study indicate there are various weaknesses in the students’ dissertations, particularly related to the way of thinking and inappropriate use of research method and statistical method. Inappropriate population, bias sampling technique, and inappropriate data analysis process and hypothesis testing are also found in the


INTRODUCTION
The ability of writing an academic paper plays an important role in deciding the educational success of a person, starting from basic education level through university level.At the university level, writing an academic paper has become the requirement of one's graduation.
The increasing ability to write an academic paper for every university student becomes very important to develop academic thinking skill (Bizzell, 1992).To those who will become an expert in a particular field, academic writing ability is very crucial, including writing and publishing article, writing and publishing book, writing and presenting conference paper, and conducting scientific research (Hyland, 2002).
In fact, most of the students in the bachelor, master's and doctoral degrees programs in Indonesian universities, including those of the Universitas Pendidikan Indonesia's School of Postgraduate Studies, are having difficulties in conducting research, reporting the research results, and writing an academic article based on the research.The observation result shows many of them are unable to do apply writing behaviors, so the ability in doing and reporting research, researching, and writing an academic article still does not meet the expectation.
Writing a thesis and dissertation is definitely such a tough process to be through that even most of the writers call the process as a "roller coaster journey" (Roberts, 2004).With the exact ups and downs in each phase of the writing processes, writing indeed becomes a difficulty for college students.Data showed that more than half of college students in the US who took doctoral degree ended up with a drop out (Cantor, 1993;Roberts, 2004).They had finished all the subjects, but they could not accomplish the last requirement of the doctoral program.Based on this fact, the phrase ABD (All But Dissertation) appears for those who could not finish their dissertation (Brause, 2000;Roberts, 2004).
The success of the college students who study at the School of Postgraduate Studies (SPS) of Universitas Pendidikan Indonesia (UPI), especially those who undertake doctoral education program is determined by the accomplishment of writing a dissertation.The quality of a dissertation does not only increase self-impression of the graduates, but also the institution.One of the strategic efforts that have been made by UPI, in this case, is forming a Commission of UPI's SPS, whose job is to review dissertations, give recommendations to the School of Post-graduate Studies, and assess students' Exams.
The main duty of the Commission is to do an assessment on the substantial, methodological, and technical aspects of the dissertation as well as making recommendations for the doctoral students who are supervised by the supervisors (promotors).From 2012 to 2013, the Commission had done a very professional job and reported various minor errors and weaknesses of the dissertations reviewed.The Commission's findings became important information for identifying every aspect of students' dissertations' weaknesses.Based on the information, the present study made a review and analysis of the quality of the doctoral students' dissertations.
The main focus of this research is on the problems related to the use of research method and quantitative data analysis in the students' dissertations which were written based on the quantitative approach.Previous research on a similar topic has been conducted by Fitt, Walker, and Leary (2009) and Fitt (2011), focusing on the literature review in doctoral dissertations in Instructional Technology.The objective of this research is, therefore, to discover the common weaknesses and strengths of the dissertations written by students as well as their influencing factors.The research is done to the dissertations reviewed by the Commissions from 2012 to 2014, employing quantitative research.

LITERATURE REVIEW
Writing any text, including academic writing, reflects the creativity of the writer (Roberts, 2004;Glatthorn & Joyner, 2005;Kamler & Thomson, 2006;Thody, 2006).The creativity should be constructed by finding ideas, assuming the problems until the project is starting, and writing an academic article.Writing is not only about picturing the process of writing ideas in the form of words, but also picturing the "entire event" of research processes, starting from initiating an idea to zero drafting (Bolker, 1998).
Generally, a thesis or a dissertation should have a relevant literature review, the past research paper's description, the upcoming result as well as discussion, and conclusion and suggestion for the next research (Phillips & Pugh, 1994).According to (Phillips & Pugh, 1994) The research method is a part of the content that demonstrates the research validity.Research method describes methodology, content, and research procedures in details (Hamilton & Clare, 2003:12).The research methodology is important for illustrating a comprehensive, consistent, and accurate procedure of research, in order to give the other researchers a replication about the way data was analyzed (Calabrese, 2006).The researcher reports what was happening on the research, including: Settings ( the place where the research took place); Participants (research correspondents); Methods (research method); Data collection (how data were collected), and; Data analysis (how data were analyzed).
The research methodology includes primary data or voice/textual reports of research methods and secondary data from other sources to justify that the method is suitable to the research and another trusted research (Thody, 2006).According to Hamilton and Clare (2003), research methodology should have research design which covers research plan; research participants, and how they are selected; ethics approval (if necessary); detailed procedures of the research data; and how data are collected, processed, analyzed, and constructed.
Research result is the main part of a piece of research.Here, a dissertation with or without research will be easily differentiated (Allison & Race, 2004).In this essential part, a researcher is supposed to be concerned about the background problem, hypothesis, and theoretical literature behind the research.It is mainly needed to integrate and relate the research results, background theory, and research methodology.That is why, research result is essential in drawing a conclusion so that the whole arguments of the research are relevant (Thomas & Brubaker, 2000).
The objective and function of the research result are to demonstrate the quantitative data and findings as comprehensively as possible.It is essential to initiate the research data planning.The data have to be crucially based on what is happening in the research as the consequence of the research procedure used.
Data ought to be presented logically and be understandable for the reader.Data are mainly the recapitulation of the image.When data are presented systematically, they give clear information to the reader.Therefore, the ways of presenting data are based on the way data are collected or based on the research questions (Thomas & Brubaker, 2000;Paltridge & Stairfield, 2007).Evans and Gruba (2002), Hamilton and Clare (2003), Glatthorn and Joyner (2005) stated that, first, data should be presented in a clear and comprehensive way by orderly arranging the data according to research problems; for example, research questions, hypothesis, research objective or research theme; presenting the data by subheadings; and using visual aids such as table, figure, diagram, model, or graphs, diagram, model, graph for summarizing data.Second, data should not be confusing.Once data are presented, the writer should make a comment and integrate the data with previous research.Third, a successful dissertation has these following attributes: there are well-presented table/s and figure/s, comprehensive data on the important points, logically presented data, and systematic data arrangement.Fourth, a good dissertation should sufficiently put the data on the appendix so the reader, who would like to, will know how data are collected.For example, the researcher saves the interview transcripts in the appendix to show the relationship with the summary of the interview.
Research discussion is the most important aspect of the dissertation, written accordingly based on the research result.Research discussion is the comprehensive explanation of background theory of literature review and the whole research.According to Thomas & Brubaker (2000), research discussion is the further explanation of theory and implication of the research result for an evaluative and integrated result.Research discussion refers to the interpretation and significance of research results.Data show what is happening in the research, and discussion shows the meaning behind the collected data.Data sum the effect, while discussion explains the effect (Hart, 2005).Data are a required aspect of answering the research questions.
Explaining the research data is the most difficult thing college students would think.It is because the tension of rational part and creative part of the brain will be very clear during comparing the research result and hypothetical theory (Evan & Gruba, 2002).Despite the data being presented, this part is more evaluative and integrative at how good the researcher supports the hypothesis to answer the research question (Thomas & Brubaker, 2000).
At the end, a dissertation has become the monumental piece of someone in his/her professional career.This is why assessing the quality of dissertations has been tremendously discussed.Pros and cons about the indicators of a quality dissertation cannot be avoided.Lovitts (2005) has done research from 2003-2004 involving 276 lecturers from 74 departments with 10 different disciplines in 9 universities in order to arrange the categories of dissertation, namely outstanding, very good, acceptable, and unacceptable.
The arrangement of the indicators is actually to show the expectation of the lecturers.What practitioners should subsequently do is translating the expectations into a form of rubric of criteria of each dimension of dissertation in detail.These are the references being used for analyzing the dissertation quality in this study.

METHOD
The research applied a qualitative approach, focusing on the process and meaning.Through this research, qualitative information from descriptive analysis can explain the tendency, ways of thinking, and irregularities of the dissertations.Constructivist philosophy, on which the study is based, assumes that facts show the plural dimensions, interactions, and exchanges of social experiences.
The analytical descriptive design was used in this research, which involved ontological activities.This research emphasizes the real picture of the dissertation in order to support the data presentation.The qualitative design is considered more appropriate because dissertation quality is essentially something plural and multi-perspectives.This design can show the relationship between the researcher and the respondents to find the meaning of the research.
The data as the proof are the main part of the study.Yin (2011) explained that data are the smallest or lowest entities or recorded elements resulting from some experience, observation, experiment, or other similar situation.Therefore, data is the smallest element of the experiences, observations, trial and error.
Based on the definition, data in the forms of documents, situations, and occasions were assessed by the Commission of UPI's Post-graduate Studies.Consequently, the main data resources of this qualitative research are comprehensive words, actions, and additional data as complement.The main data resource is the reviewers of the Commission of UPI's Post-graduate Studies.The secondary data are the reviews of 106 dissertations written by doctoral students of the study programs of: Social Science (6), Guidance and Counseling (12), Bahasa Indonesia (8), English Language (5), Curriculum Development (7), Out of School Education (9); Education Administration (27), Natural Sciences (7), Physical Education (1), Mathematics (9), General Education (10), Value Education (2), and Technical and Vocational Education (3).
In accordance with the main objective of the research, the data collected are ultimately focused on the quantitative research methodology used in the dissertations.This means the data were mainly related to research design, validity, experimental design, and analysis.The analytical data usually are related to the hypothesis testing, assumptions, and research result presentation.
The researchers in this qualitative research act as the human instrument who explore data and select facts and information as needed for adaptability, responsibility, knowledge, and capability to handle the sensitivity, to identify the problems, to clarify and arrange the summary, and to explore and analyze the special response (Cohen, Manion, and Morrison, 2007).The researchers were involved directly in the reviews of the dissertations provided by the Commission.Table 1 displays the instrument used as the analytical guidance on the use of research methodology in dissertation using the quantitative approach with hypothetical and/or inferential statistics.
The analysis was done using the model of interactive analysis (Miles &Huberman, 1984).This model has three components, namely data reduction, data presentation, and verification.These components are interactive and continuous, forming a cycle.The analysis procedures can be explained as follow.
Data reduction: The researchers resume, consider the essential things, focusing only on the important parts, and find a theme and pattern of the weaknesses of the dissertations found by the Commission.The researchers arrange the data at a glance.Data presentation: In this component, the researchers reduce the in-hand information from the first procedure and present the narrative texts based on the common weaknesses of the dissertations found.In this procedure, the researchers only arrange the data at a glance.Verification: The researchers elaborate the research focus in detail and do deeper analysis until the conclusion about the common weaknesses of the dissertations is drawn.The debatability of the statistical data analysis method used.The weaknesses found in this area include: Improper data analysis technique, incomplete data analysis, irrelevancy between hypothesis formulation and statistical hypothesis, too many hypotheses, irrelevant statistical method, the use of unrealistic research methodology, weakness in data analysis technique related to the research question(s), improper hypothesis testing technique, unclear contribution of the research result, research on model instruction is not completed with the explanation of the essential model, improper statistical assumption, statistical assumption is only normality and linear testing when non-linear testing is needed.In addition, other errors found include: At the testing of double regression and structural equation modelling (SEM), the statistical assumption is not properly applied which makes the requirement wrong: multicollinearity, heteroscedasticity, autocorrelation; improper use of variable analysis; elaboration of the hypothesis testing is used to differ the correlations, determinations, and coefficient analysis; improper decision about hypothesis testing using double regression and path analyses; and insufficient knowledge about regression testing and path analysis.

DISCUSSION
The research results show that the definition of research methodology is mostly given in the dissertations using quantitative research method; however, the dissertations mostly do not explain about what, how, and why the methodology is used.In addition, the postgraduate students are unaware of the research approach and methodology, so they misunderstand the definition.Research methodology refers to how the formation and procedure of the research are.Broadly speaking, research methodology is the procedure of how to organize the problems in order to know the information as the solution of the problem.Therefore, the methodology is an overall scientific procedure to find the solution of a problem.With those procedures, one can conduct the research as a continuation of the previous research methodology with similar objects and subjects and with the identical result.
Related to the research objects and subjects, units of data analysis, population and sample, sampling technique, and the research results, the postgraduate students do not properly differentiate between the research objects and subjects.These are considered as a weakness in the writing of the dissertation.Research object is the variable to observe (Borden and Abbot, 2002).The observable phenomenon is represented by a concept or variable which is general and uses a certain testing or measurement as a continual process.Variables are the important components in data presentation, problem questions, and representation of the certain concept and variable.Some postgraduate students do not completely understand the object and subject of their research.In deciding the units of analysis, the students are unable to differentiate the units of analysis of individuals, groups, or organizations.For example, the unit of analysis of a piece of research is supposed to be at organization level, but the data are taken from the individuals or vice versa.This kind of error makes a dissertation becomes meaningless and rejected for national or international publication.
Meanwhile, the research subjects relate to the persons who are being investigated and the individuals as the units of analysis.The formulation of a research problem has to consider how to decide the units of analysis, for it will be related to the research design, data collection, and data analysis decision.The research is related to the perception or behavior of how data are taken, namely from individuals, groups, institutions, or organizations.The data collection is supposed be clear and relevant to the research participants.
The unit of analysis is how the element is analyzed or the research element to be comprehended for a certain problem.The research subject is generally understood as individual, family, group, organization, and formal and informal structure.In research on organizational behavior, for example, there are three levels of analysis, i.e., individual level, group level, and organization level.The unit of analysis is important to draw conclusions.Hence, when it is mistakenly decided, the conclusion becomes inappropriate.
Related to data resources, the weaknesses are found in how the students fail to provide reasonable explanations for selecting certain data resources.Data resources should be related to the units of analysis.In deciding data resources, the students are frequently unaware of the percentages of the samples.
Population can be persons, or groupd of people, organizations, things, or objects.Related to this, weaknesses are discovered in the definition of the population and the sample.Ideally, the population is the target of generalization.The population should be specified entirely.Units of a population should be relevant to the specification set by the researcher(s).On the other hand, the sample refers to a part of the population realistically taken by using a certain technique.Thus, to select the sample, theoretical definition should be discussed based on the operational definition Obviously, the sampling problem can be solved using random sampling, but the post-graduate students in this research often assume that if random sampling is done, the research is basically on the track.Meanwhile, to decide the sampling technique, a certain condition is required, such as research budget or other important considerations.
In considering sampling technique, it is basically based on the probability or non-probability sample.It is based on the extent to which its generalization will go through and the research goals to be achieved.The probability sampling technique is used for the representative sample to a wider generalization, while the non-probability sample is used if the time, budget, and other components are to be considered instead of the generalization.
Another important aspect of quantitative research is the statistical data analysis as in regard to answering the research problem and testing the hypothesis.The statistical method is chosen for understanding the research data that leads to a proper conclusion.
The research has found that the data analysis technique is misused.It is due to a lack of awareness in deciding the statistical method, the validity of research formation, measurement scale of the variable, and power-efficiency testing.Hamilton and Clare (2003) showed that research procedure is differentiated by causal research, that is the research that presents causes from certain problems (experimental or nonexperimental study).Correlational research is the research that presents essential variables that correlate other variables, and comparative research is the research that shows the difference of one group with another group of certain variables.
Statistical data analysis technique should be relevant to the research plan.The researcher is supposed to be aware of the research formation, sampling technique, and type of hypothesis, also the theory on which the research is based.It is essential to integrate and correlate the research result, descriptive result, and the hypothesis test.Therefore, it is essential to acquire the conclusion so that the arguments of the research discussion will be relatively connected with the analysis (Thomas & Brubaker, 2000).
The study implies the need for improving the quality of the dissertations written by UPI postgraduate students by improving dissertation supervision, developing the capacity of the supervisor in quantitative research and academic writing and publication.Hence, the students will be able to conduct quantitative research, report it, and write academic article(s) publishable by any reputable journals.
It should be noted that the main weakness of this study is a lack of generalization due to the limited sample involved and the case study method employed.It is suggested, therefore, that the study needs to be repeated in other cases and/or in a wider scope by employing more samples selected from several universities, so the typical weaknesses in the postgraduate students' dissertation can be identified.

CONCLUSIONS
Some of the weaknesses found in the dissertations written by UPI's post-graduate students which use quantitative research are found in how the researchers decide the research methodology and the reasons why they choose it.The weaknesses are also found in defining the research population and sample, considering the units of data analysis, using the sampling technique, defining sample size, and applying proper data collection technique(s).The statistical hypothesis testing and data analysis technique are found to be improper, too, as well as irrelevant statistical hypothesis, inappropriate hypothesis testing, ineffective contribution of research result to the theory, and improper use of the assumption of hypothetical testing.
The final target of writing a dissertation is not only reporting research, but also writing for academic publication in any accredited national journal or in any indexed international journal.The findings confirm that improvement in dissertation writing is needed.Therefore, it is recommended that the School of Postgraduate Studies of UPI assign the supervisors of dissertation with expertise relevant to students' dissertation topics, as shown also by the supervisors' track records in academic publication.Dissertation supervisors are also encouraged to always improve their knowledge capacity in terms of research methodology, quantitative data analysis, and supervisory service.The students, on the other hand, should also improve their capability in doing research and be eager to learn from articles published in the academic journals.