Assumptions underlying online learning

One of the fundamental assumptions of all training (except perhaps on-the-job training) is that people have the ability to port contexts. In other words, they can see and understand and get good at something in a training context, and can then apply that learning where it is supposed to be applied -- in the different context of their work. This ability to transfer is probably uniquely human -- you can show Fido videos of someone throwing a ball for a dog all day, but when you then throw the first real ball for him he'll look at you as if you are mad.

In an online mode, unless you are teaching computer-related skills, perhaps we call for a greater leap of the imagination than in a classroom mode. Everything we see and do is in two dimensions on a flat screen. Making the leap from the truly virtual online world to the totally real workplace world may be heroic for some. Or maybe not -- the past few years have seen such a surge in computer proficiency that maybe people feel more comfortable than they did before.

On a more practical note, there's a bunch of assumptions. Sometimes we make them, other times we are smart enough not to assume, others we design around.

We often assume that online learners have a lot more ability to concentrate than they do in reality. We assume a relatively quiet interruption-free learning environment. We assume that they have the will and the motivation to push through the work without a classroom pace-setter. We assume people want to work at their own pace in a time convenient to them, rather than being led by someone else's schedule. We assume that learners are comfortable with all of the alarming personal privacy issues that online learning potentially exposes them to. We tend to assume a particular technology platform for online learners (operating system, browser version, plug-ins, bandwidth, performance) that in my experience is often way off the mark. We assume people read e-mails. We assume people want to take the courses they are taking. We assume either that they need to be led down a predetermined path mapped out by an instructional designer, or that they want to be able to find their own way. We assume that they want to finish what they started. And we assume that all of the unscripted "fuzzy learning" that fills up the white spaces in the classroom activities is either irrelevant or somehow handled better online.

Oh, and we assume that it all works, technically and pedagogically, because nobody tests software fully before it gets to the customer :-)

Revealing the deeper talent set

Most large corporations are proud of their "human resource" or "human capital" management abilities, yet I have so often found that very little is known about the qualities, experiences, interests, and talents that exist within those resources. People are hired to do a particular job, only the relevant qualifications become memorialized somewhere in a HR database, and that becomes who they are, the only face the company sees.

Back in the 1980's there were movements to create more transparent, multidimensional databases of personnel, particularly among multinationals who were always moving people from one country to another. But even those systems logged only the dimensions that the company thought relevant.

Tip of the iceberg is a good way to look at it -- what we see of each employee is only the bit that we care to shine a light on. Yet the talent pool in every company is probably a hundred times more valuable than the surface value of those assets. We seem to skim the surface, looking only at the familiar or the apparently relevant. This lapse is systemic, and it is personal. How many employees feel that their immediate manager undervalues them, dismisses their not- immediately-applicable qualities and experiences, and looks only at their immediate field of expertise? How many of us can remember all the stuff that our direct reports put on their resumes back when we hired them? And anyone applying for a job has only a couple of pages in which to describe their qualifications, so usually leaves out the rest of the iceberg anyway.

We will pay a couple of hundred dollars for a royalty-free stock photo, or we'll lay out a couple of thousand to commission a photographer to shoot something specific for us. But how many gifted photographers are there already on the payroll? How many musicians, artists, writers, linguists, web-designers, technicians, videographers are hiding within the mild-mannered accountants and call-center staff? And if we are looking for management or marketing or leadership, how many experienced and passionate community leaders, volunteers, mentors, lobbyists, activists, organizers, are secretly leading double lives as programmers or line workers?

Seems to me that, without invading anyone's privacy, some of the emergent social networking tools might help a company find out what kind of resources it REALLY has at its disposal.

Writing skills training

If you need help with this, I suggest you contact CDI (Communications Development Inc.)-- Brad Olander is the person to talk with (tel. 202.721.0350 ). CDI is a very competent and experienced group specializing in training report writers in clarity and structure. They do classroom work and have something online as well. They do a lot of work for operations like the World Bank, and they are local to the DC area.

Blogs and RSS

Like most things these days, the blog is not a static beast -- the ways the tools are used are evolving all the time, and one year from now today's blogs will probably look quaint.

There are lots of places to go to get started, and the blognoscenti provide plenty of links. A good basic RSS streamer that can get you up and running in minutes is Amphetadesk.

Download it for free

Training Haiku

Several years ago, someone ran a competition for "haiku computer error messages" that produced some real gems that I still remember:

Yesterday it worked
Today it is not working
Windows is like that

Everything is gone;
Your life's work has been destroyed.
Squeeze trigger (yes/no)?

It would be fascinating to see some training/learning/development/OD/PIC haiku.

Learning is asking.
Which wrench is the right one
to hammer in the nail?

Blogs and RSS

many of the concepts and functionality that enable blogging have been around in one form or another for a long time.

What may be new, though perhaps not revolutionary, in the context of "knowledge power" is the ease with which blogs allow ordinary human beings (hopefully with real knowledge and experience) to publish their thinking to a broad audience. You don't need to be an HTML expert or a Webmaster or a content management guru or a geek of any kind to get your ideas out there in, literally, minutes. This is what good technology should be like - invisible to the user.

The added benefit of RSS is that if you want to benefit from the fresh content of a particular blogger without going to his/her site several times a day, your RSS feed will bring you the updates. You can also use RSS as a kind of always-on search engine which looks for new dynamically-served stuff inside all the RSS-equipped blogs on the Web, and serves it to you when available. Ordinary search engines are really poor at finding dynamic content, really slow to acknowledge it, and quite hopeless at making it readily available.

Surviving info explosion: Journalism, blogs, and workplace learning

Journalism and T&D both ride the same wave. Both try to gain clarity in their field of knowledge, which is an increasingly difficult thing to do. The explosion of knowledge (yes, much of it junk info) is a phenomenon that trainers, learners, and journalists have to find ways to filter. Learners often turn to trainers to help them stay ahead of their game, not only in providing relevant raw knowledge but also in helping to learn how to apply it in the development of skills.

The information explosion is not a myth. According to a UC Berkley study published last year, more information has been produced in the past five years than in the total of all of previous human history. An Exabyte is a billion Gigabytes. We produced 23 Exabytes of new information in 2002 -- double that produced three years earlier. That's 3.6 Gigabytes per person per year. Much of it is fog, much is ephemeral and never stored, and a big chunk of it (400,000 Terabytes in 2002, or 40 times the Library of Congress print collection) is in e-mail, and nearly as much again is in Instant Messaging. Even looking at more 'solid' information (that actually stored in some medium or other), in 2002 we produced 800 Megabytes per member of the human race.

While blogs add to the fog, many also cut through it. Journalists are supposed to highlight relevant important information, provide analysis and tell us what it means, and (sometimes) what we should do about it or how we can apply it. Trainers, in a corporate world beset by rolling change, are often expected to play a similar role. I wonder, though, about the role of educators. Should not one of our educational priorities be to teach people how to deal with this information explosion, how to learn at an accelerating pace for life? Or should we be teaching people to STOP PRODUCING ALL THIS STUFF.

As an aside, and not wishing to be too Zen, if I write a blog and nobody reads it, does it really exist as information?

I'm going to buy stock in Johnson & Johnson, or whoever it is that makes Tylenol.

Blogs - what's the big deal?

Some see blogs as no different from personal Websites, and decry their textiness. Blogs are not Web sites, communities, FAQ boards or newsgroups. They are a little different in many respects. They may be fads in that the way Web-log tools are typically invoked today is unikely to be the way they are invoked tomorrow. But these days the capacity for rapid evolution is a key to viability. The inherent contradiction of techno-fads is that built-in rapid-redundancy is how they stay alive.

And don't knock text on the Internet. E-mail (text) is still the biggest single application of the Internet. Text in books, newspapers, and magazines is still quite common too, and most of us do not refuse to read a book because it has no pictures. Text may not be cool, but it's a pretty good tool for communication.

People will read what appeals to them. If a blog is unsophisticated and tries to target a sophisticated audience, it's not going to get much traffic. And, as with anything you see on the Web, read in your local newspaper, or hear on a press conference, you have to be ready to use your own cr@p-detector.

As a 'serious journalist' (i.e. one who makes a living primarily from traditional media), John Dvorak's perspective on blogs is bound to be a little patronising, or even defensive. But, apart from the masses who put up short-lived vanity blogs (e.g. those that publish their own poetry), Dvorak acknowledges that there are some very useful blogs around in the public domain. Closed-group corporate blogs are even more relevant and useful, but you are not likely to ever get access to them.

Technology and training

New technologies are being used in T&D, but like most technologies they are used to a fraction of their true potential. I don't mean that the "obvious" and simplistic application of the technologies should be more widespread (heaven forbid), but that vastly more creative or intensive intelligent application of existing technologies is possible. "E-learning" on the whole is regarded as pretty dire not because of technology but because it is all too often invoked in an uninspired mundane or lazy fashion. We are awe-struck by new things, get slightly more familiar with them, then don't want to venture beyond that comfort zone. Look at e-mail or blogs: the technologies that deliver them are capable of much more sophisticated, friendly, user-focused communication, yet we are content to live with "this".

Mobile Internet access, particularly cell-phones, excite me. Anytime anywhere used to mean anytime you were at a computer linked by a dial-up modem to an ISP. Now it means while standing in line at the post office. What we DO with that connectedness is a blank slate. I have a personal interest in the use of the Internet as one of the tools to help economic development in Africa, a notoriously infrastructure-devoid part of the world. The 1990's concept of the Internet assumed desktop computers linked to landlines or cables, few of which exist in Africa. But cellphones are much more widespread and affordable. A "computer" is no longer a gray box on a desk -- it's a cellphone or something like it. And the Internet is not restricted by wires -- it's in the air around us. The implications for education, teacher training, teaching support, small business development and so on are significant.

In the context of learning, and using 'product' loosely, I'd say the exploitation of data-mining and search technologies is the most under-rated product today. Even in e- commerce it is not used anywhere near its potential. We mine lead and cast dull gray ingots. It is still a primary industry. We have the potential to know so much more about individuals and their interests or abilities and to respond much more acutely. Yet we continue to ask the wrong questions, aggregate the answers, and McDonaldize our deliverables.

Some of the most important skills a trainer can have today are the ability to transcend yourself and appreciate and respond to the contexts and concerns of others; the ability to rapidly unlearn or discard that with which you are comfortable; and the ability to learn or adopt or invent new things: concepts, attitudes, processes, methodologies, paradigms, cultures, contexts. Vital also are the judgment to know the difference between a fad of no real value and important new thinking, and the ability to appreciate that the fad often has a longer half-life than the significant evolution.

Values as a trainer?

I don't think your honesty and integrity as a trainer are bent by the profession you come from, though they may have a bearing on the profession you choose.

I have to say that in my experience some of the best trainers (and the most honest) are those with “real-world” experience outside of their training experience. Seasoned practitioners, subject matter experts, or those who have recently completed their own training, are often very astute, very learner-aware, very passionate, and very attuned to the context of their teaching.

I have spent twenty five years in training and development (in one way or another), but I am a marketing professional and always will be. I got into training largely I think by accident, out of a passion for the relevant subject matter and extreme irritation at the way it was a) taught by the training professionals and b) practiced by the practitioners. So I suspect there is lurking within this unassuming exterior a crusading evangelist somewhat obsessed with impacting actual real-world performance.

I first put on a trainer's hat while I was still an idealistic business student -- as head of AIESEC South Africa, I had a mission to improve the relevance and quality of business education in universities throughout the region. When I started a real job as a market research analyst/consultant with A.C. Nielsen, I found myself teaching my peers how to do a more valuable job for their clients, which led to me teaching analytical marketing to Nielsen clients like Unilever and Revlon, which led me to become education director for the South African Market Research Association and to found and run an industry certification program.. At the same time, my passion for doing my job -- sales and customer service -- got me into teaching my colleagues basic skills such as presentations, sales, and consulting. Some major OD work resulted from that, which got me transferred to running Nielsen's international management services center to replicate the work world-wide. Since then I have run my own training/consulting/OD businesses in various parts of the world, focusing mainly on sales, marketing, and technology.

All of which background poses the question: am I a trainer or a consultant? Or am I a salesperson, or a marketer, or a researcher, or an entrepreneur? And does it matter? I believe I approach my work with a level of customer- focus, diligence and integrity that is unusual among “training vendors” and my clients would agree. I think I was born and raised to be that way. But, if my lifetime in consultative selling has influenced me, it has amplified – not diminished – my integrity.

Boring PowerPoint? Blame not the tool.

It's not PowerPoint or the blue background that causes slumber, but badly structured, poorly communicated content that may in itself be of little value to the audience. Why we prefer to blame the tool rather than the presenter's inept use of it is beyond me.

We have all read (or partially read) many really badly written books, books with poorly structured content, books that are simply uninspired. But you don't hear people say that books are dull, or snicker about the narcotic effect of black text on white paper. Except perhaps in the context of e-learning where "page turner" has acquired a meaning quite the opposite of its meaning in a book context.

Global villages and global conflict

Marshall McLuhan "global village" has not happened as he envisaged. Where's the homogenization of cultures and values that was supposed to happen with the globalization of knowledge?

This has not happened, nor will it, partly because knowledge in itself is impotent without context, and geographical/cultural contexts are enormously diverse. Global knowledge is interpreted in a local context, and its meaning changes accordingly. So cultures become enriched and further evolve down their path, rather than mutating to mimic the source of the knowledge. (This is one of the challenges of e-learning -- while many vendors like to chant the trite cliché that content is king, without an acknowledgement of context the learning will often be ineffective, or at least may achieve results that diverge from those intended).

What is also happening, though, is that the globalization of knowledge (despite attempts to manipulate mass media) is causing individuals, groups, and movements to be smarter and more aware of their context in relation to the rest of the world. This not only reinforces a sense of unique identity, it brings about an (often subconscious) evaluation of their satisfaction with the status quo. You can't covet your neighbor's cow if you don't know he has a cow, if you don't know what a cow is, or if you don't know who your neighbor is. You can't think of yourself as materially better off than or morally superior to a foreigner if you have no idea what that foreigner's circumstances are. You can't comprehend the extent of your oppression or freedom, wealth or poverty, unless you are aware of alternatives to your condition. So, far from bringing about global harmony and homogenization, global knowledge can bring about increasing tensions within and between different cultures.

Anti-American protesters often wear US-iconed clothing, and feel no contradiction. We see the irony because from our perspective foreigners should not be allowed to reject an American policy without rejecting all other things American. In our simplistic view of the outside world, anyone who embraces one aspect of our culture while rejecting another cannot be taken seriously. The demonstrator sees no irony because in his context there is none. Lots of kid's who demonstrate against "America" are more sophisticated than we give them credit for -- or at least they are able to differentiate between the issues that they protest and the things that they like or respect. If kids in America protested something that the US administration was doing, would they have more credibility if they boycotted American goods and services too? I think not.

Culture and technology adoption

The adaptation and frequency of use of various technologies is based to a great extent on national cultural factors. But I think that while adoption of a technology may -- at times -- be ubiquitous across all members of a national culture, it's intensity of use is often very sporadic, with "power usage" being limited to a few demo/psychographic groups. In the US we all have microwave ovens, but I am willing to bet there are very few "power users" of them. We all have cell phones, but only certain segments of the population use SMS "texting" with any level of comfort. We all have the potential to IM, but some use it constantly, others turn it off. We all have VCR's but many of us can't even program them to record. It seems that everyone has an SUV, yet nobody ever goes off-road.. In the US at least, the national culture is often "gotta have it" rather than "gotta use it"; we consume the tools but not their applications.

It is true that corporate culture, when exported, often does not overcome local culture. Where I have found corporate cultures that appear partially at odds with (my perception of) local culture, it has rarely been because of some value system or behavior norms exported by a foreign parent. More often, it has been a strong set of norms inculcated within a company by one or more strong, influential leaders and his/her acolytes. Corporate culture is based on a bottom-up foundation, but it is molded by top-down examples. Respect for punctuality and respect for those whose time you waste when not punctual is either a strongly manifested corporate value or it is not -- no matter where in the world you may be.

Culture and punctuality

I have found in most cultures (including the US) that time has different states of focus depending on context. In group social engagements, unless there is an external delimiter (such as the movie starts at 3:00), the focus is very fuzzy -- nobody shows up at 8:00 sharp for a party. In one-on-one social engagements, the focus tends to be a little sharper depending on circumstances. In one-to-one or one-to-many business engagements, the "one" in the driving seat tends to have a much sharper focus than do individual members of the "many" -- a trainer likes to start on time, but the trainees are late; a salesperson likes to be punctual, but the potential buyer(s) have no problem keeping him/her waiting.

Our perceptions of what is "normal" behavior are determined by the habits of our most familiar peer groups. Over the years I have done a lot of work in various Latin American and Asian countries, South Africa, and most of Europe. In a business meeting context, the sensitivity to punctuality is always less cultural than contextual. And within that context you cannot make sweeping statements about national cultural attitudes to time because corporate culture plays a major role in guiding those attitudes. There are a couple of companies that I have worked with in Mexico and Brazil where I am always the last to arrive at a meeting that I am running. Conversely, there are companies in the US and UK where I have given up expecting more than 50% of participants to be punctual, and where participants come and go at will throughout the meeting. Generation and social set also plays into time consciousness. The post-MTV perpetually-looping CNN mode of communication has produced some people who have not only a limited attention span, but who assume that there IS no beginning or end, and believe they can always catch up no matter where they start or how often they get distracted. That fragmentation of attention span seems to be getting worse with the advent of SMS.

Is Internet time overwhelming national culture? As with the CNN comment above, I don't see it on a national scale. The digital divide (if we think of it in terms of those who have embraced connectedness v. those who just get by) is just getting wider. True, "smart mobs" can coalesce and disperse with split second precision. But these are funky folks on the fringe, not mainstream people. What may become more pervasive, particularly as mobile phones become smarter, is a blurring of the line that separates "presence" from "absence". Perhaps technology will be used to inflict punctuality -- you have an electronic window of opportunity to enter this meeting and if you miss it the virtual doors will be closed on you. Or perhaps the technology will make "punctuality" an unnecessary outmoded concept.

Should training ever be mandatory?

There is no absolute on this issue. Like all things in training and education, "it depends" on a host of factors.

The fact that some people vociferously criticize the training they were mandated to attend cannot be taken as valid criticism of the training itself. The majority of people probably go into mandated training with some resistance, if they are not pre-sold on the goals/benefits. Those most resistant will usually find fault with the training -- the trainer, the content, the context, the learning design, will all come under fire. Others will come out of the experience enlightened and appreciative.

Maybe our inherent dislike for being forced to participate in training is a throw-back to our school days. How many of us appreciated mandatory attendance of classes at school? How many of us ran down the teacher, the content, the relevance of the learning, just to rationalize our desire to be elsewhere? Where schooling is not mandatory, a nation declines. The same may be true of training in a company. A corporation has goals, the achievement of which require certain competencies. Each employee cannot be expected to automatically align his/her personal training needs with those of the company, and there are bound to be fundamental disconnects.

This is particularly true in times of change, and it may well be that those individuals most resistant to change are the ones least likely to participate in required training voluntarily.

Sample size and statistical significance

How many people do you need to survey to get a significant result?

For statistical significance (in statistics, "significant" has a very specific meaning), you need to use a valid sample size. You also need to use a valid methodology for selecting who goes into your sample.

As a rough rule of thumb, your sample should be about 10% of your universe, but not smaller than 30 and not greater than 350. If you are doing multivariate analysis, the sample should be ten times the number of variables you are testing.

If you want to be more pedantic, you should define what confidence level you want and what margin of error is acceptable to you. A confidence level of 95% and an error margin of 5% tell you that your result will be within 5% of the true answer 95% of the time you run the survey. So if you tested 100 samples, 95 of them would return a result that was within 5% of the truth.

The correct sample size is a function of those three elements--your universe (how many people make up the group whose behavior you are trying to represent), your desired error margin, and your preferred confidence level. It's a simple formula (well, not so simple). For most purposes, I'd go for a 10% error margin at 95% confidence. For varying numbers of learners in your universe here are the ideal sample sizes (the first at a 10% error margin, the second at 5%):

50 in the universe, sample 33 or 44
100 in the universe, sample 49 or 80
200 in the universe, sample 65 or 132
500 in the universe, sample 81 or 217
1000 in the universe, sample 88 or 278
and so it goes till you find that an ideal sample for a 10% error margin hardly moves above 350 no matter how big the universe (it's 500 for 5%).

Course licensing

If you are licensing courses, you should find that the vendor is flexible about meeting your needs. Typically, you will buy a license for X number of enrollments (an enrollment being one learner in one course), and that license will have a time delimiter as well as the maximum enrollment delimiter. So, for example, you would license a course or package of courses for one year for a maximum of 400 enrollments. The effective cost per enrollment will drop depending on volume. Usually, you buy the license in advance, and it is a use-it- or-lose-it agreement.

You can also license on a pay-per-use basis, paying only every time a new enrollment takes place. Expect to pay a fixed cost per enrollment that is substantially higher than on a volume license. You would only opt for pay-per-use if you had very few learners -- course fees are relatively expensive, and you and the vendor incur more administrative load.

There are many in-between license negotiations that you might be able to do. A common one is to buy a small volume license with an agreement that when you exceed its ceiling you can roll into a second discounted license, or into pay-per-use at a discounted rate. Also, if you are interested in a collection of courses, you can buy different volume licenses for different courses depending on your expected usage, and opt for pay-per-use on those courses that are likely to be used by small numbers.

The key is to go into a negotiation knowing what your usage is likely to be, and get your vendor to work with you on building a package that works. Off-the-shelf courses should not have inflexible off-the- shelf license terms :-)

RoboDemo v. Camtasia

RoboDemo is certainly more powerful (feature-filled) than Camtasia, so it is harder to learn. But it is a great product. If you are planning on serving your Flash over the Web then RoboDemo is probably a better bet -- not just because it runs easily with your LMS. The screen motion capture system in RoboDemo is fundamentally different to that of Camtasia. Camtasia records the entire capture area for each frame, whereas RoboDemo captures only what has changed in each frame (and frequently this is just curser location). So RoboDemo's files are considerably smaller than Camtasias. I have also found Camtasias post-capture edit facilities to be rather crude.

RoboDemo will probably do what you need, and a lot more, and it will be relatively quick to learn. Buy it before Macromedia assimilates it and doubles its price.

Graphics

When you need graphics or pictures in designing training, do you buy or make your own?

We typically do both: We buy royalty free stock photos and take some photos ourselves or commission someone to take them where necessary, depending on the need. We also create our own graphics or outsource to a contract graphic artist. Typically we will sketch out what is required and have a designer create the finished work. We would always outsource video work.

I use Illustrator and Photoshop a lot, because there is not much that you can't do with them and they work together well. For animated stuff, a combination of Premiere, Camtasia, Flash, and RoboDemo seems to cover most in-house needs. Learning curves are not too steep on Adobe tools if you are not trying to reach maistro level. For that we outsource, often offshore.

I can also recommend one of the best services on the Web - elance. Elance is like a marketplace for freelancers, many of them in Asia. You post a project description and a budget, and people bid on it within minutes. You don't pay till you are happy with the deliverable, and for a limited graphics project your credit card rarely takes more than a $50 hit. We have often posted a project online (e.g. turn these sketches into Flash animations, yesterday!) and had it executed perfectly in only a few hours. BTW, you can get anything from a simple drawing to a complete Website, from programming help to a complete software solution. Larger organizations could learn a huge amount about customer service and responsiveness from the individuals we have dealt with through Elance. Elance.com is the kind of networking that the Web was made for.

For stock photos, we usually start with corbis or StockMarketPhoto. The "completely free" graphics sites usually don't have much that appeals, and (IMHO) much of the free stuff does your end-product no favors in credibility terms. It's often the online equivalent of those irritating ant-people (were they called ScreenBeans?) that PowerPoint abusers used to stick everywhere a few years ago.

For self-shot photos, I normally use a Nikon 950, though I have contractors who use both higher-end and lower-end cameras. I recommend shooting at the highest resolution available, and then cropping and scaling down resolution in Photoshop. You use 72dpi for Web photos, and in outputting jpegs and gifs you need to try to keep them down to file sizes of less than 20-30kb -- though if the need warrants you can go up to 200kb or more. We try to keep total graphic load on a page to under 80kb, which is still rather fast on a slow dial-up. If your target audience is all on broadband, you have a lot more leeway. Photoshop lets you play with image optimization, and tells you the file size and load time while you are tweaking.

Critique of E-learning Consortium's research methodology

This study is full of holes, and the industry deservs better. Others have challenged the research design, pointing out that the sample is self-selected. But it is not just in sample selection that the original study about drop-out rates was flawed. The questionnaire design itself was flawed to the point where no analysis could produce meaningful results. When the rfesearcher posted his original solicitation for participation in the first study on the trdev list, I responded in some detail with my concerns (relevant sections are at the end of this message **).

Research that is not "statistically valid" in any pure sense is very common in the commercial world, and is used often where time and budget and study goals do not make "pure" research a sensible option. For each situation there is an acceptable research instrument that will best satisfy the need. That's fine, and usually those commissioning the work are aware of any limitations. But I share many peoples' concern that we are seeing more and more of what is effectively "pop-quiz" material being misrepresented to a wider audience as serious research with meaningful results. There is enough confusion and ambiguity in the e-learning world as it is, without such studies adding to the mythology.

** Extracts from my response of 6/3/03:

>From the questionnaire, it seems that little useful information can come from the information being asked for. I have objected in the past to "research" data that is meaningless because of its lack of specificity, the inability to identify and cross-tabulate causal data, or the dubious design of the sample. This study is no different. E-learning is not defined for respondents, yet respondents are required to answer questions about it, without themselves being asked to define what they mean. True, later in the questionnaire respondents get to list their own e-learning experiences, but the ambiguity and breadth of the list provided is astonishing - here it is:

* CD-Rom
* Live instructors with online materials
* Facilitated online instruction
* Self-paced online without facilitation
* Real time virtual classroom
* Video
* Other

The nature of the list aside, there is no way that data analysts will be able to meaningfully connect reasons for abandoning courses (selected from an earlier list) with even the broad type of experience selected from the list, because it is all "select all that apply". Far better to have asked respondents to pick one instance of abandonment and asked questions about that. Now that may have produced some useful numbers that may help us all do a better job.

All that can possibly come out of this is "X% of e-learners abandoned at least one e-learning course at some time" and -- as a completely separate data point that can not be mapped back to the drop-out figure -- "the most common reasons cited for abandoning are..". Garbage in, garbage out. But I am sure that the data will be spun in all sorts of fascinating ways for presentations and press releases, and some numbers will find their way into training mythology. The aggregated answers to the following will probably get a lot of play, too:

* My estimate of the average rate of completion for e-learning in my organization is (select a percentage range).

What does this question mean? Rate of completion could be percentage of people who never drop out, percentage of courses completed, percentage of people scheduled to take courses who have already finished them, percent of total company training hours that has so far switched to e-learning... And, more importantly, what qualification or data does the average respondent have to substantiate his/her estimate?

In the past, such studies have sought to determine the percentage of COURSES that are abandoned, where this seems to be seeking the percentage of LEARNERS who have ever abandoned a course. I would imagine that to be 100% (that video aerobic dance program that I started five years ago really didn't get me hooked -- yes, video is classified by the study as e-learning). And if it is learners, not courses, that are the focus of the study, should we not be trying to find out more about them than a few basic demographics?