ePortfolio: Dr. Simon Priest

RESEARCH & STATISTICS

 

 

THE RESEARCH INQUIRY PYRAMID: The Research Pyramid (1994) explains my philosophy regarding inquiry. It describes a sequence of steps that I believe researchers ought to take when investigating a phenomenon (called "IT" here). Answering a good research question leads to more good research questions. Therefore, I always recommend that future inquiry direction should be built upon past evidence-based inquiries. Otherwise, researchers would take a scattered (shotgun) approach to investigation and much time, money, and other resources would get wasted.

I like to use the example of earthquakes or cancer to explain this concept. If we attempted to predict and control these (causality and discrimination points where studies are currently focused), without understanding influence, relationships, differentiation, or description, then we would most certainly fail. However, by initially reporting what these phenomena look like, how they are similar to other geothermal events (volcanic eruptions) or other diseases (failed immune systems), what else is associated with them, and what can impact them, then we build a strong foundation to inform our inquiry toward stabilizing cities or finding a common cure.

 

 

POSITIVISTIC vs. NATURALISTIC PARADIGMS: In 1987, I was a devout positivist (deductive and objective scientist) and my then life partner was a confirmed naturalist (inductive and subjective artist). Her 1988 dissertation conducted three studies: a statistical analysis of quantitative data collected under positivism, a thick and rich description of qualitative data on the same topic conducted under naturalism, and a comparison of the knowledge generated by both approaches seeking "truth" from evidence. She won a top award for her ground breaking research. Together, we developed this comparative table by debating philosophy around the dinner table and on long dog walks.

 

 

BRIDGING QUALITATIVE / QUANTITATIVE GAPS: I grew up in Vancouver and this photo of the Lions Gate Bridge over the First Narrows is a great metaphor of bridging the gap between quantitative and qualitative data. Regardless of one's affirmed paradigm for conducting research (Naturalistic or Positivistic), bridging is the same.

Researchers are like photographers. The Positivist photographs wildlife and frequently chooses a telephoto lens to get in close to their subjects. The Naturalist photographs scenery and often selects a wide angle lens to take in the whole view. Since the photographers seek to create different images, they pick different lenses. As researchers, they ask different questions and therefore gravitate toward using different methods. The Positivist prefers to measure quantity, while the Naturalist prefers to describe quality.

However, sometimes the wildlife photographer will use the other (wide angle) lens to gain a broader perspective of the entire herd and ocassionally the scenery photographer will switch to the other (telephoto) lens to capture a particular object of interest. Researchers are the same. Despite their predetermined research philosophies and inquiry paradigmal preferences, many are deciding to utilize both types of data in the same singular study.

The content added within this attractive picture lists the key hallmarks of each data type: its name, definition, formats, sources, examples, and analysis techniques. The key factors that may bridge the gap between them are listed in the middle. MEDIUM sample size is the first opportunity. Since large randomized samples are more common in quantitative and small purposeful samples fit better with qualitative, the middle ground of medium size enables the use of both data analyses. MIXED methods can be sequential (one before the other or in reversed order) or convergent (both in parallel, then joining together). Since one approach tests hypotheses and the other creates theory, doing BOTH can be a common focal point for the same study. Data can be converted from one type to the other. Triangulation involves using other data or one data set to confirm and corroborate another. Longitudinal designs offer the chance to combine methods as both can examine changes over the long term. Irrespective of beliefs, researchers can partially integrate these two methodologies by many means.

 

 

THE TWELVE V’S OF BIG DATA ANALYSIS: I started working with big data in 2008 and extrapolated some extra V-words in 2010 to describe successful characteristics of big data management.  Consider these explanations, where data is the plural form of the singular datum.

With ongoing improvements in data capture and storage, and the advent of the “Internet of Things” (where most devices and appliances are networked), enormous amounts of information are being generated.  A size limit (ex: 1 Terabyte or larger) does not determine the classification of big data sets.  Instead, big data are defined as any information set that cannot be analyzed by traditional methods or statistics.  Big data require a uniquely pragmatic approach to their gathering, secure storage, and analysis.

The purpose of analyzing big data is to make predictions, often about human behaviors.  For example, epidemiologists use big data analysis to spot patterns in the spread of disease.  Economists use this to identify business or market trends.  Meteorologist and climatologists use it to forecast weather and track changes in global temperatures.  Each of us benefits from big data analysis when we use a map app to plan our route to a destination.  The movement of every GPS-enabled cell or mobile phone is recorded and analyzed to determine traffic patterns (slowdowns and gridlocks) that are then fed back to mapping applications on those same phones, so we can choose the best route to follow in rush hour.

In educational administration, I’ve spearheaded big data initiatives for my institutions at four levels of analytics: tracking uncompleted assignments, poor academic performance, and lack of contribution to team tasks (learning/teaching level), real-time aggregate presentation of student enrollment, retention, probation, and expulsion (academic/administrative level), effective sources of marketing information correlated to new registrations (executive/institutional level), and measuring e-learning quality and adaptive learning success to influence policy and accreditation standards (government/legislative level).  Nevertheless, higher education has fallen well behind other sectors in big data analytics and applications, due to an extreme shortage of qualified staff.

Analysis of big data can avoid sampling and simply consider the full data set.  Alternatively, a set can be sampled.  The former, analyzing the entire population, does not need to adhere to the usual statistical assumptions of parametric procedures (used to estimate the sample’s representation of the whole population).  While the latter, sampling some of the population, definitely does require statistics and comparison with the normal curve remains compulsory. 

Four V’s have been coined to describe the characteristics of big data: volume, velocity, variety, and veracity.  To these I added eight more to expland the considerations of managing big data: virality, value, viscosity, variability, validity, verifiability, viability, and visual.  This wheel diagram summarizes the twelve V’s of big data.  

  1. Volume (quantity): can you find what you are searching for in the generated sea of data?
  2. Velocity (speed): are data captured and processed rapidly enough to meet decision demands?
  3. Variety (type/nature): do the collected data portray the breadth and depth of the construct?
  4. Veracity (quality): are the data informative or disinformative and do they impact accuracy?
  5. Virality (dispersion): can outcomes be rapidly circulated and decisions placed into action?
  6. Value (usefulness): are the data and their analysis/predictions helpful for making decisions?
  7. Viscosity (resistance): is data flow unrestricted or bottlenecked in the analysis processes?
  8. Variability (inconsistencies): are errors present and do these hinder the overall analysis?
  9. Validity (precision): do the data measure the construct and variables they purport to measure?
  10. Verifiability (reliability): are the data consistent representations and truthful measures of reality?
  11. Viability (sustainability): Is the data gathering going to last or will predictability remain feasible?
  12. Visual (image/idea): are the analyses outcomes or predictions easy to understand and present?

 

 

ELEMENTS OF E-RESEARCH AT THE EXTREMES: Electronic Research or eResearch means conducting evidence based inquiry (research) in an online (electronic) setting through reliance on information and communication technologies (ITC).  While eResearch can be as simple as conducting a survey online, I am excited by eResearch at the extremes, where collaborative thinking, cloud or fog-based grid-computing, and big data sets are engaged.  Since I proselytize that the current solutions to the world’s great problems (water shortages, food imbalances, climate change, peace, poverty, etc.) will come from crowd-collaboration through technology, I am keen to conduct eResearch and support faculty in their efforts to do so, as a natural fit with this necessity and eventuality. 

The key difference between current or future eResearch and present or past traditional inquiry is the use of distributed locations.  The eResearchers are found in varying geographic places.  Big data are acquired from dispersed sources and stored in the cloud (spread across many server locations).  The analysis takes place via grid-computing networked across scattered sites.  Collaborative discussion takes place over ITC.  Findings are published and disseminated through the Internet (email), on websites, in electronic journals , and via web conferences.  Traditional inquiry saw researchers working in isolation or small teams, keeping data on their own servers, conducting analysis on their own computers, not collaborating openly, and disseminating findings via print journals and in person conference attendance.

Successfully carrying out eResearch at the extremes requires the inquirer to consider nine interactive and important elements.   Based on my past efforts in eResearch, these nine are organized into three groups of three (research conduct, cyberinfrastructure, and other resources) as shown below.

 

 

TECHNOLOGY TRANSFER AND COMMERCIALIZATION: In my academic administration and business leadership roles, I’ve played various parts of the technology transfer and commercialization (TTC) sequence from researcher and evaluator, through protector and investor, to marketer and developer.  In working with innovative software or training, invented facilitation techniques, improved models or theories, and imaginative fashion with wearable technology, I created a spiral to explain the TTC sequence.

Technology transfer involves the dissemination of new knowledge or information, tools or machines, and methods or processes from the original point of innovation, invention, or improvement to people, places, or other parties.  For example, a new way to test for a disease, discovered by researchers using university resources, needs to be further developed within a joint venture that fairly includes the inquirers, their institution, and other funding sources. Commercialization involves distributing this transferred content to a sales market where profitability is possible.  Continuing the above example, once fully developed, the disease test is mass produced for consumption by hospitals and medical labs.  Eventually, a home version may be created and sold “over the counter” to the average customer. Commercialization sometimes follows technology transfer, but these can also occur independently of one another.  The spiral sequence describes the two together.

  1. Discovery: invention, improvement, innovation, idea
  2. Disclosure: intellectual property (IP), full description,  specific detailed criteria
  3. Evaluation: viability, feasibility, expert feedback (distinct/sustainable competitive advantage)
  4. Protection: IP ownership, copyright, patent, (with clear liability/accountability)
  5. Formation: licensed startup company, with optional franchises, and corporate partnerships
  6. Marketing: commercial analysis, market research, business plan (unique elements of distinction)
  7. Development: build and test prototype, scalability, ongoing refinement, production
  8. Revenue: distribution, sales, profit sharing, reinvestment, benefits to society, growth

In the research phase, discoveries are made and intellectual property (IP) is disclosed to the sponsoring institutions.  In the safeguard phase, those institutions work with the researchers to evaluate the likelihood of success and move to protect their IP.  In the organize phase, a company is formed to take the IP further,  market the IP, and develop strategic plans for commercializing the IP.  In the commercialize phase, the IP is further developed and sold with revenues being shared and reinvested in additional research as the TTC sequence repeats.

 

 

FIVE KINDS OF PROGRAM EVALUATION: Most of my students get confused about program evaluation and mix everything together into a single amalgam. In order to understand the five kinds of program evaluation, one must first understand the five phases of program development, because evaluation is conducted at different places in this developmental sequence. Jude Hirsch and I created these phases (2002), building on earlier work I did with Lee Gillis and Mike Gass.

  • DIAGNOSE: Assess need by multiple methods/sources (use combination of conversations, surveys, interviews & observations)
  • DESIGN: Plan program based on assessed needs/desires (refine purpose, goals & plan LETS: logistics, events, timing, staff)
  • DELIVER: Present program as planned, but be flexible (include intro, body of events, conclusion & remain open to changes)
  • DEBRIEF: Facilitated discussion around events (facilitate by funneling, front-loading, framing, freezing, focusing & fortifying)
  • DEPARTURE: Closing activities before clients leave (action planning, evaluations, anchors/souvenirs & schedule follow ups)

 

 

COMPARISON OF THE FIVE KINDS: As you can see in the table below from the year before, each type of program evaluation (and its "also known as" names) fits into the sequence at a different location and for a different reason (2001). The methods used are the same as in social science research with descriptive data analysis. This isn't rocket science, but you do need a map to avoid getting lost.

 

 

RESOLVING ETHICAL DILEMMATA: Dilemmata (plural of dilemma) are choices to be made among multiple alternatives, where no single selection is the favorable preference and all options are apparently undesirable.  This happens often in life and I use this approach to reach resolution in those dilemmatic times.  However, when conducting research with humans or animals (in whole as subjects or in part as tissue samples), I have the additional responsibility of ensuring that my chosen methodology does not harm them. 

While the decision making and problem solving processes are familiar (see multiphasic methods), these five approaches inform one’s judgment, when drawing on previous experience is unavailable, by providing data for the unknown in times of uncertainty.  By definition, a dilemma is rife with uncertainty and the unknown.  Thanks to Mike Gass and Jasper Hunt for their clarifying time spent chatting about ethics as adapted from Kitchener’s (1984) model of moral reasoning.

The approach takes the form of signposts.  I like this metaphor because, on those rare occasions where I have been lost while route finding in uncharted territory with map and compass or modern technology, I am thrilled to discover a sign on a post that points me in the direction I want to go!  For me the signposts are, at best, salvation from disaster or devastation and, at worst, reinforcement for what I already know in my heart or mind to be true.  Signposts offer a series of steps arranged in sequence and, most of the time, resolution can be reached by only considering their first few signs.  However, in a more complex dilemma, the last few signs become necessary and are thought provoking and reinforcing. 

Basically, these five steps are shown and described below.  I follow them in sequence and, when resolution becomes obvious, I stop at that point.  However, sometimes, I go all the way to the end to reach resolution, where the compromising choices are more about whom or what gets harmed the least and whether I could defend making the same choices in similar situations.  I have always passed these five approaches on to my research students and other faculty.  I hope they have found them useful.

 

 

THE FAMILY TREE OF STATISTICS: When I was working on my doctorate, I had the great pleasure to support Lorraine Davis, teacher extraordinaire of statistics. I learned a great deal from her, but the greatest gift she gave me was the opportunity to be her teaching assistant and a lead instructor of the lab sessions that complemented her lectures. This forced me to know my statistics really well and struggle to be as good as her at teaching them.

While working together on a small handbook of statistics in 1985, we evolved this family tree to show how some of the more basic tests related to one another. The arrangements below are rather self-explanatory.

 

 

THE CONFIRMATION MATRIX FOR STATISTICS: Something Lorraine shared with me, early on in our conductor passenger relationship, was a basic confirmation matrix as a double check to make sure one is using the correct test. I expanded it to its present content in 1986 and have applied it over the years since then. The key to using this matrix is knowing your data type. Its format (for both independent and dependent variables) can be: INTERVAL (numbers in sequence that can have equivalent differences among values), NOMINAL (has no sequence and therefore no numerical difference among values) or ORDINAL (numbers in sequence, but the difference among values cannot be determined). When necessary, interval data can be converted to ordinal data, but not the reverse. Simply find the corresponding list of tests from both data formats and then triple check against the family tree above.

To help understand the types of data consider that you and I are in a 10K race, where I finish second to your first place. With just this much information, we don't know how close or far apart we were when finishing. These are ordinal data. If we knew our run times, then these would be interval data (where we can quantify the gap between us). Obviously, interval data (times) can be converted to ordinal (placement), but not vice versa. Nominal refers to classification variables like gender (male/female) or brand of running shoes (Adidas, Nike, etc.) and these nominal data cannot be turned into interval or ordinal, because nominal has no numeric worth or order.

 

 

TRANSDISCIPLINARY INQUIRY: Students often confuse multidisciplinary, interdisciplinary and transdisciplinary terms, so I made this graphic in 1995 to help them see the evolving difference. Late one night, I was explaining to a doctoral student my thoughts on the difference between basic and applied inquiry with respect to interdisciplinary and transdisciplinary inquiry of complexity with diversity. I drew this diagram of four purposes for inquiry on a napkin; here is a better version from that same year.

 

RESEARCH INSTRUMENTS: Here is a collection of instruments that I authored or co-authored in the 1990's.

General Surveys

virtualteamworks.com

Interpersonal Trust

TEAM EFFECTIVENESS ASSESSMENT MEASURE: LONG (100 items, 30 minutes, single use) & SHORT (50 items, 15 minutes, repeated use)
VIRTUAL TEAMWORK (50 items, 10 minutes): SELF (use for self) & OTHERS (use for team members)
measures INTERPERSONAL TRUST on 6 dimensions:
A=Acceptance,
B=Believability,
C=Confidentiality,
D= Dependability,
E= Encouragement, and
O= Overall Trustworthiness
for GROUPS, PARTNERS, SELF & ORGANIZATIONS
LEADERSHIP EFFICACY ASSESSMENT DIAGNOSTIC: LONG (100 items, 30 minutes, single use) & SHORT (50 items, 15 minutes, repeated use)
ELECTRONIC FACILITATION (50 items, 10 minutes): SELF (use for self) & OTHERS (use for team members)
PRIEST ATTARIAN RISK TAKING INVENTORY: BUSINESS (10 scenarios, 30 minutes, use pre- and post) & OUTDOOR (10 scenarios, 30 minutes, use during)
GLOBAL LEADERSHIP (50 items, 10 minutes): SELF (use for self) & OTHERS (use for team members)
DIMENSIONS of the ADVENTURE EXPERIENCE: LONG (26 items, 15 minutes, once daily) & SHORT (2 items, 1 minute, multiple times)
ONLINE TRUST (50 items, 10 minutes): SELF (use for self) & OTHERS (use for team members)
s i m o n _ p r i e s t @ y a h o o . c o m
Copyright © 1975 | All Rights Reserved