This is a response to Brad Feld’s recently published book Startup Communities, which is summarized in a 3.5-minute video here.

Feld’s “Boulder Thesis” is highly aligned with the research on fostering effective communities of practice. However, based on that literature, Valid Eval (VE) believes there are opportunities to help startup communities take advantage of an under-utilized resource—the tacit expertise of their community’s “entrepreneurial stack.”

VE was founded as a way to begin capturing the wisdom of the crowd for the mutual benefit of the startup community. VE began this process by providing novice entrepreneurs (newcomers) with reliable feedback from more experienced community members (veterans)—taking advantage of historically tested learning models rooted in mentoring and apprenticeship.

VE’s evaluation strategy is grounded in cognitive science research on the social construction and cultural distribution of knowledge. Cognitive scientists, like John Seely Brown (Figure 1), have compared community knowledge to an iceberg, divided between the explicit knowledge of individuals (and artifacts) and the tacit knowledge of groups (and their cultural practices). This knowledge is unevenly distributed, with the vast majority of a community’s expertise residing in the hidden world of social groups and their practices.


Figure 1: John Seely Brown’s View of Knowledge

This inequitable knowledge distribution is particularly formidable for newcomers to a practice—constrained by access, most of what matters remains hidden from their view.

VE began tackling this problem by focusing on what Brad Bernthal described in Feld’s book as the “glorious mess” of business plan competitions. Similar to the motivation behind TechStars, VE believed there must be a better way for experienced entrepreneurs to help those getting started. Most business plan competitions rely on rank-and-order scales to make decisions, providing limited to no feedback to entrants who dedicated significant time to their submissions. VE began addressing this problem by using performance rubrics to make the expectations valued by startup veterans visible to newcomers.

VE worked closely with experts to develop rubrics that surface, prioritize, and capture performance criteria tacitly used by veteran community members to make decisions. VE’s mission, aligned with the Boulder Thesis, is to make these privately held assumptions highly transparent, open, and accessible to entrepreneurs new to the process. Newcomers use the rubrics as a shared frame of reference to begin with the end in mind, chart their progress over time, and navigate risk through self-assessment. Interacting with the rubric, as a cultural artifact, entrepreneurs begin to align their decisions more closely with community stakeholders and participate more fully in their practices.

While designed to support newcomers, VE has found developing rubrics to be mutually beneficial for community veterans as well. In order to develop a rubric, experts involved in their development must externalize knowledge hidden even to themselves after years of experience; a dynamic known as the paradox of expertise. Creating a rubric forces veteran community members to articulate values and priorities in ways that place them in the role of ‘learner’ and away from that of ‘knower’. Additionally, once externalized, rubrics optimize the time and expertise of community veterans and volunteers when mentoring. Embedding common feedback in a rubric frees veterans to provide even more customized recommendations about what they believe matters.

VE’s evaluation software currently adds value to startup communities by providing entrepreneurs with reliable and trustworthy feedback from veteran practitioners. VE believes this service is critical to both newcomers and veterans of a startup community. Entrepreneurs benefit by having more data from community insiders to make better, more informed decisions. Veterans benefit by being forced to regularly explicate and test their theories on what matters and how to most effectively provide feedback. In this way, startup communities begin to generate consensus around cultural practices, values, and priorities.

VE’s plans for software development involve tapping an additional reservoir of tacitly held community knowledge—the expertise embedded in the horizontal exchange of information between peers. Like Feld, VE believes the informal connections made between peers and near-peers (those slightly ahead in the startup process) are an invaluable resource within the entrepreneurial community. VE is eager to develop mechanisms to surface, capture, and track the collective feedback of one’s peers over the developmental trajectory of a startup.