Validated Learning Quarter

Validated Learning Quarter (hereinafter, the Quarter) is a lecture introducing the learners to product discovery primarily through key topics related to validated learning. The Quarter is the first of four lectures of Business Quadrivium, which is the second of seven modules of Septem Artes Administrativi (hereinafter, the Course). The Course is designed to introduce the learners to general concepts in business administration, management, and organizational behavior.

Outline

Concept Management Quarter is the predecessor lecture. In the enterprise discovery series, the previous lecture is Idea Generation Quarter.

Product discovery is the

Enterprise discovery

enterprise discovery of data needed to design or modify market exchangeables that enterprises offer on the market. Organizationally, the data needed to design or modify these market exchangeables is collected through idea generation, validated learning, monitoring, and market intercourses. This particular lecture concentrates on validated learning because these enterprise efforts are the primary method for collecting data that emerges as a result of product developments.

Concepts

1. 

   Validated learning
   Validated learning. The acquisition of knowledge through experience generated by trying out an idea and then measuring it against potential consumers to validate the effect. Each test of an idea is a single iteration in a larger process of many iterations whereby something is learnt and then applied to succeeding tests.
   • Action research. A change process based on systematic collection of data and then selection of a change action based on what the analyzed data indicate.
2. Learning. Any relatively permanent change in behavior that occurs as a result of experience.
   • Lessons learned. The learning gained from the process of performing the project. Lessons learned may be identified at any point.
   • Lessons learned process. A process improvement technique used to learn about and improve on a process or project. A lessons learned session involves a special meeting in which the team explores what worked, what didn't work, what could be learned from the just-completed iteration, and how to adapt processes and techniques before continuing or starting anew.
3. Social-learning theory. A theory of learning that says people can learn through observation and direct experience.
   • Social learning.
   • Social teaching.
   • Learning-by-doing.
4. Elicitation. Evoking or drawing out data from someone in reaction to one's own actions or questions. As an enterprise effort, elicitation most commonly consists of two phases: (a) to identify data sources and (2) to use elicitation techniques (e.g., facilitated workshops, interviews, observations, artifact testing, etc.) to gather data from those sources.
   • Intelligence elicitation. Elicitation of intelligence information from people as opposed to automatic surveillance.
   • Expert elicitation. The synthesis of opinions of subject matter experts on a subject where there is uncertainty due to insufficient data.
   • Prank elicitation. Elicitation of intelligence information from people using a practical joke or mischievous act.
5. 

   Concept vs experiment
   Experiment. A procedure carried out to support, refute, or validate a hypothesis. Experiments are commonly undertaken in order to make a discovery, test a hypothesis, or demonstrate a known fact.
   • Pilot experiment (also known as pilot study, pilot project, or, simply, pilot). A small-scale preliminary experiment conducted in order to approximate feasibility, time, cost, adverse events, and improve upon the feasibility study design prior to performance of a full-scale research project.
6. Reconnaissance. In the military, the exploration outside an area occupied by friendly forces to gain data about natural features and enemy presence. In enterprise administration, the exploration of the market to gain data about features of the market and competitor's behavior.
   • Observational reconnaissance. Reconnaissance that utilizes observation, sometimes using assumed personas.
   • Experimental reconnaissance. Reconnaissance that utilizes experiments, especially purchasing rival market exchangeables.
7. Surveillance. Close observation, especially of a suspected spy, criminal, or rival.
   • Targeted surveillance (or targeted interception). A form of surveillance that is targeted on specific persons of interest.
   • Mass surveillance (or bulk interception). A form of surveillance that targets places rather than people. Mass surveillance is routinely accused of treating innocent people as suspects in ways that are unfair, of violating human rights, international treaties and conventions as well as national laws and of failing to pursue security effectively.
   • Human surveillance. Surveillance conducted by human beings, often by human operatives, and/or on human beings. Human surveillance can be a part of human resource intelligence.
8. Testing. Taking measures to check the performance or reliability of something, especially before putting it into widespread use or practice, or somebody, especially before hiring. Someone who or something that conducts testing is called a tester. Someone who or something that is tested is called a testee. If a testee is a human being, this testing is called human-subject testing. The testing that utilizes one or more artifacts such as a prototype or market exchangeable is called artifact testing. The testing that utilizes no artifact is called natural testing.

   Testing	Tester
   	Human	Machine
   Natural	Observation, oral examination, open voting, interview, etc.	Measurement and signature intelligence, automatic data validation, etc.
   Artifact	Questionnaire survey, ballot voting, user test, etc.	Computer-based exam, online survey, etc.

   • Verification. Testing a product at a given stage of development to ensure that it meets specifications and requirements. Verification ensures that the solution is developed correctly; it answers the question, "Did we build the system right?" Also see requirements verification.
   • Validation. (1) Testing a market exchangeable to ensure that it satisfies its intended use and conforms to its specifications and requirements. Validation ensures that the correct solution has been developed; it answers the question, "Did we build the right system?" Also see requirements validation; (2) Testing a developed product to insure that the created system actually provides the value intended to its stakeholders.
   • Pilot. Any test implementation of a service with a limited scope in a live environment.
9. Concept artifact. An object experimentally made to visualize and/or test a concept especially as a result of the preparative or investigative procedure.
   • Wireframe. A sketchy representation of a prototype. Wireframes are commonly developed in order to arrange elements of a future system. For instance, a wireframe can serve as a rough guide for the layout of a website or app, either done with pen and paper or with wireframing software.
   • Mockup. A model of a design for a market exchangeable developed or to be developed. Mockups are commonly used to test graphic designs. If a mockup possesses any degree of functionality, it is is considered to be a prototype.
10. Prototype. A partial or preliminary conceptual model of a deliverable developed or to be developed; this model is used as a reference, publicity artifact, or data-gathering tool. A prototype allows measuring if an product idea attracts interest.
    • Low-fidelity prototype. A quick and easy translation of high-level design concepts into tangible and testable artefacts, giving an indication of the direction that the product is heading.
    • Paper prototype. (1) A rough, often hand-sketched, drawing of a user interface, used in a usability test to gather feedback. Participants point to locations on the page that they would click, and screens are manually presented to the user based on the interactions they indicate; (2) A type of usability testing where a user performs realistic tasks by interacting with a manual, early-stage version of the interface that is often manipulated by an individual who is upholding the illusion of computer interactivity. During this process, the details of how the interface is supposed to be used are withheld from the user.
    • Throw-away prototype. A prototype used to quickly uncover and clarify interface requirements using simple tools, sometimes just paper and pencil. Usually discarded when the final system has been developed.
    • Exploratory prototype. A prototype developed to explore or verify requirements.
    • Evolutionary prototype. A prototype that is continuously modified and updated in response to feedback from users.
    • Horizontal prototype. A prototype that shows a shallow, and possibly wide, view of the system's functionality, but which does not generally support any actual use or interaction.
    • Vertical prototype. A prototype that dives into the details of the interface, functionality, or both.
    • High-fidelity prototype. A prototype which is quite close to the final product, with lots of detail and a good indication of the final proposed aesthetics and functionality.
11. Minimum viable product (MVP). A version of a new product that includes sufficient features to satisfy early adopters and allows a team to collect the maximum amount of validated learning about customers with the least effort.
    • Wizard of Oz minimum viable product (WoOMVP). A version of a product that looks functional, but it actually operated by a human behind the scenes, granting the appearance of automation.
    • Concierge minimum viable product (CMVP). A manual service simulating the same exact steps people would go through with a final product.
    • Piecemeal minimum viable product (PMVP). A functioning model of a product that takes advantage of existing tools and services in order to emulate the user experience process.
12. Independent review. A formal assessment or examination of a market exchangeable, process, business, enterprise and/or proposed change, accomplished partially or completely by someone or someones outside the group that has developed the product, runs the process, business, or enterprise, and/or proposed the change.
    • Inspection. The independent review that is based on careful examination or measurement of something in order to either learn about its features or check whether its features confirm its specifications or specific requirements. Inspection can be accomplished by a customer or third party. Inspection can also be a formal type of peer review that utilizes a predefined and documented process, specific participant roles, and the capture of defect and process metrics similarly to structured walkthrough.
    • Audit. A planned and documented activity performed by qualified personnel to determine by investigation, examination, or evaluation of objective evidence the adequacy and compliance with established procedures or the applicable documents and the effectiveness of implementation.

Roles

1. Tester. A stakeholder responsible for assessing the quality of, and identifying defects in, a software application.
2. User. A stakeholder, person, device, or system that directly or indirectly accesses a system.
   • End-user. A person or system that directly interacts with the solution. End-users can be humans who interface with the system, or systems that send or receive data files to or from the system.
3. Actor. The human and nonhuman role that interacts with the system.
   • Secondary actor. An actor who participates in, but does not initiate a use case.
4. Human operative. (1) An individual skilled in working with people including covert gathering of intelligence data; (2) A practitioner of human resources intelligence.
5. Interviewer. An individual who interviews persons by telephone, mail, in person, or by other means for the purpose of completing forms, applications, or questionnaires. He or she asks specific questions, record answers, and assist persons with completing form. He or she may sort, classify, and file forms.
6. Mystery shopper (secret shopper).

Methods

1. Elicitation technique. An established procedure for gathering enterprise data from human beings. Elicitation techniques are used in anthropology, cognitive science, counseling, education, knowledge engineering, linguistics, management, philosophy, psychology, and other fields. A person who interacts with human subjects in order to elicit information from them is called an elicitor. The most common techniques include interviews, brainstorming, focus groups, artifact testing, observation, and questionnaire survey.
   • Requirements workshop. A requirements workshop is a structured meeting in which a carefully selected group of stakeholders collaborate to define and or refine requirements under the guidance of a skilled neutral facilitator.
   • Needfinding. Needfinding is the art of talking to people and discovering their needs; both those they might explicitly state, and those hidden beneath the surface. It is only in truly understanding people that we can gain meaningful insights to inspire and inform a final, impactful design.
2. Interview. An elicitation technique that represents an arranged meeting of people face-to-face or remotely, in an informal or formal setting, in order to elicit information from a person or group of people by asking relevant questions and documenting the responses. Those interviews that are conducted for consultation or informational exchange can be called informational interviews.
   • Structured interview. A planned interview designed to gather job-related information.
   • Unstructured interview. A short, casual interview made up of random questions.
   • Open-ended interview. Covers a variety of data-gathering activities, including a number of social science research methods.
3. Stakeholder interview. A conversation with the key contacts in the client organization funding, selling, or driving the product.
   • User interview. Used for understanding the tasks and motivations of the user group for whom you are designing, user interviews may be formally scheduled, or just informal chats.
   • Focus group. Small (usually, 5-15 individuals) and composed of representative members of a group whose ideas, attitudes, or opinions are sought. By asking initial questions and structuring the subsequent discussion, the facilitator or moderator can elicit enterprise data. A focus group may discuss a specific product, process, market, solution, project, and/or enterprise practices, related risks and estimates in an interactive group environment. Guided by the facilitator or moderator, the participants are asked to share their impressions, preferences, needs, use practices, responses to management regulations, etc.
   • Panel survey. Involves the random selection of a small number of representative individuals from a group, who agree to be available over an extended period - often one to three years. During that period, they serve as a stratified random sample of people from whom data can be elicited on a variety of topics.
   • Remote survey. A survey administers a set of written questions to stakeholders in order to collect responses from a large group in a relatively short period of time.
4. Artifact testing. The data-gathering technique that is based on taking measures to check the performance and/or reliability of somebody, especially before making agreements, or something, especially before putting it into widespread use or practice.
   • Black box test. A test written without regard to how the software is implemented. These tests show only what the expected input and outputs will be.
   • Acceptance test. The derivative from the acceptance criteria that verifies whether a feature is functional. The test has only two results: pass or fail. Acceptance criteria usually include one or more acceptance tests.
   • Usability test. A user sits in front of your website or app and you have them perform tasks and think out loud while doing so.
   • Contextual inquiry. Interviewing users in the location that they use the website or app, in order to understand their tasks and challenges.
   • Diary study. Asking users to record their experiences and thoughts about a product or task in a journal over a set period of time.
   • Unit testing. A short program fragment written for testing and verifying a piece of code once it is completed. A piece of code either passes or fails the unit test. The unit test (or a group of tests, known as a test suite) is the first level of testing a software development product.
   • User research. Observation techniques, task analysis, and other feedback methodologies which are used to focus on understanding user behaviors, needs, and motivations.
   • Alpha test. Controlled internal testing of a pre-production model, intended to detect design flaws or functionality deficiencies.
   • Beta test. External pilot-test after Alpha testing is complete and prior to commercial production. In beta testing, the product is released to a limited number of customers for testing under normal, everyday conditions in order to detect any flaws. (see 10 Experiments To Test Your Startup Hypothesis)
5. Event-powered survey. The data-gathering technique that is based on a systematic study of behavior of people at arranged events such as pooling, sampling, and/or querying, either virtual or physical, undertaken in order to gather data primarily of the results of their behavior.
6. Heuristic. Any approach, technique, and/or practice that utilizes one or more experiments to discover knowledge and/or to solve a problem.
   • Fail-fast. The heuristic that encompasses (a) starting work on a task or project, (b) obtaining immediate feedback, and then (c) determining whether to continue working on that task or take a different approach or, in another word, adapt. If a project is not working, it is best to determine that early on in the process rather than waiting until too much money and time has invested.
   • Trial and error. A problem-solving heuristic, which represents repeated, varied attempts to solve a problem continued until either success or stopping trying.
   • Learning through failure. A heuristic to utilize experience for learning.
7. Marketing data gathering. In addition to all-source intelligence.
   • Concept testing. To test the acceptance of a concept by target consumers.
   • Coolhunting (also known as trendspotting) – to make observations and predictions in changes of new or existing cultural trends in areas such as fashion, music, films, television, youth culture and lifestyle
   • Customer reaction tracking. Periodic or continuous in-market research to monitor a brand's performance using measures such as brand awareness, brand preference, and product usage in response to marketing stressors such as commercials, website promotions, store shelves, etc.
   • Customer satisfaction research. Quantitative or qualitative studies that yields an understanding of one's customer satisfaction with a transaction.
   • Distribution channel audit. To assess distributors’ and retailers’ attitudes toward a product, brand, or company. That may or may not include store audits to measure the sales of a product or product line at a statistically selected store sample in order to determine market share, or to determine whether a retail store provides adequate service.
   • Mystery shopping (secret shopping). An employee or representative of the market research firm anonymously contacts a salesperson and indicates he or she is shopping for a product. The shopper then records the entire experience. This method is often used for quality control or for researching competitors' products.
   • Price elasticity testing. To determine how sensitive customers are to price changes
   • Test marketing. A small-scale product launch used to determine the likely acceptance of the product when it is introduced into a wider market.

Instruments

1. Prototyping tool. A tangible and/or software implement used to create concept artifacts.
   • Axure. A wireframing and interactive prototyping tool, available for both Windows and Mac.
   • Balsamiq Mockups. A wireframing and interactive prototyping tool, available for both Windows and Mac.
2. Sandbox. An environment or location where experimentation is acceptable, without consequences for failure.
3. Learning management system.
   • Moodle.
4. Questionnaire. The data-gathering tool that represents a set of questions and other prompts composed for elicitation.
5. Survey form. An online form designed to solicit feedback from current or potential users.

Results

1. Findings register. An official record of a conclusion reached as a result of an inquiry, investigation, or trial.
   • Register. An official list or record, for example of births, marriages, and deaths, of shipping, or of historic places.

Practices

• Dogfooding is practice to show confidence in own product by using it; dogfooding is also another type of testing. Its name has been derived from the expression "eating your own dog food."

Business Analysis Quarter is the successor lecture. In the enterprise discovery series, the next lecture is Monitoring Quarter.