… okay, I found the article I mentioned in Amble #3, two related ones, actually. The one I’m posting here is very similar to the Introduction section in a very extensive piece called Qualitative Measurements of Organic Apostolic Paradigms. I wrote the first versions of both about the same time in 2003. Both were crafted in response to attempts by traditional-pragmatic-modernist leaders of a church-planting support network to assess “success” by their own culturally-bound definitions, instead of according to culturally-contextual definitions of “success” relevant to emerging cultures – AND in comparison with scriptural requirements.
The articles were perhaps the best summaries I had produced up to that time on an apologetic and process for changing our definitions and descriptions for success. I have edited it slightly, and added a few sections or comments [[noted in double square brackets]] on the problems of using outmoded evaluation tools or systems.
Some of this is material I’ve blogged about before, or referred to in comments elsewhere (see the end of the article for recent material). But there are some different expressions and analogies in it that may be of use. I almost changed all the instances of the terms postmodern and postmodernity to post-Christendom to shift the emphasis from a reigning philosophy to a cultural state-of-being, but decided not to. In 2003, many of us were still using the p-word; might as well keep it in for now as a historical marker, until I do a major revision of all my materials. And although this article related primarily to church planting, I think a lot of the principles apply to established churches, too.
Also, it relates in a broader sense to what may be going on with Willow Creek’s Reveal self-study, since they are looking to measure “spiritual growth” – a substantially intangible set of concepts leading to very concrete lifestyles. Similarly, measuring overall “success” and discipleship “success” in postmodern/post-Christendom cultures is far more intangible and qualitative than the concrete and quantitative version would be in modern/Christendom cultures. So, hopefully, this article will contribute to future ambles on the research methodologies of Reveal: Where are You?
Sidenote: In my usage, the terms “churches/ministries in emerging culture” and “emerging church” are not equivalents. “Emerging church” is a specific paradigm that is a few incrementals different from the established Christendom church model, but still with the same overall analytic paradigm framework primarily. (See my blog category on paradigm profilingfor details on what that means.) I use “churches/ministries in emerging culture” to signify discipleship endeavors that are indigenous to or contextualized to the newer edges of cultures in transition from the modernist era to whatever is next, and, ideally, is also countercultural to the antibiblical aspects of those new cultural edges. The newer cultural edges involve synthetic, symbiotic, and/or analogic paradigm frameworks.
Final note: Just a reminder that I am a “Citizen Culturologist.” My training is in linguistics, with significant studies and experiences in work involving social sciences, humanities, and arts. I do not hold advanced degrees in anything (including theology). I immerse myself in these subjects because I am passionate about exploring cultural systems and working to help friends contextualize ministries to be both relevant and countercultural to their settings. A professional sociologist would likely approach this differently, and that’s fine. We need different perspectives in order to get the equivalent of a “3-D theological and culturological MRI” on the subject. My work is what it is …
QUALITATIVE ASSESSMENT OF MINISTRY IN EMERGING CULTURES
Assessments: Apples, Oranges, and Fruit
- Accountability and assessments are legitimate concerns for disciple-making and church-planting efforts in established and emerging cultures.
- If modern church structures and processes are apples and emerging culture ones are oranges, it won’t work to use tests designed for apples and apply them to oranges; apple tests are not valid for oranges.
- The “validity” of an assessment relates to whether it actually measures what you say it does. Valid assessments for a specific culture arise from a deep understanding of what its adherents value and how they think; these are what lead to the surface behaviors of that culture. So, valid assessments require describing the surface practices of a culture and the churches that “work” there (i.e., are “successful”). But they also require defining underlying patterns that lead to those practices, yet distinguish one culture from another. Both apples and oranges are fruits, but what inherent features make them different from one another? And what if there are multiple kinds of apples, and oranges?
- Some systems of assessments that are applied to church plants give helpful perspectives. But they also tend to overlook important things that should be there, or are geared entirely for “apple” definitions of “success,” not “orange.” A better system would be a “fruit” assessment which could evaluate any church planting model according to its cultural context.
- If we do not figure out appropriate criteria for assessing church plants and ministry efforts in emerging cultures, we can expect that the modern/postmodern divide in ministry will remain and perhaps even widen as Christian insiders from emerging cultures are alienated from “the system.”
Validity and Reliability
Those investing in ministries to reach people in emerging cultures with the gospel are rightly concerned about whether their resources are being used wisely. They are asking legitimate questions about how to measure “effective ministry in postmodern settings.” Questions often address what are/should be our definitions of success. They address whether existing assessment tools can measure the effectiveness of process evangelism, relational discipleship, organic church planting, and apostolic process efforts in emerging culture work.
Many such questions focus on validity and reliability of results. In assessment tools:
- Validity refers to whether the assessment actually measures what it says it does.
- Reliability is whether the same person gets the same results when taking the assessment multiple times over time.
One of the major problems with applying existing assessment tools to situations in emerging cultures is that the tools’ underlying cultural, theological, and philosophical presuppositions generally don’t match with those of the cultures they claim to “measure.” So, applying pre-existing tools to just-emerging cultures is likely to result in no validity; it’s a test for apples being used on oranges. It cannot work – unless an assessment is a test for fruit and it’s being used on both existing/modern “apples” and emerging/postmodern “oranges.” However, it may prove reliable in that it consistently measures an emerging culture wrongly!
[[I witnessed an experience of this in 2007, when a church I worked with conducted a congregation-wide survey. It took me a very long time to complete it, in part because I was agitated and was trying to figure out why, at the same time I was trying to respond to the multiple choice questions. It took a while to pinpoint what was bothered me. I realized that the survey asked many questions that seemed completely irrelevant – probably geared to churches at least five times our size, or to “traditional programs” (for example, “Training Union”) that were no longer very common in this denomination. Also, it failed to ask questions that were very relevant to the world at large and in the church right now. The instrument seemed designed for a time when most mainline and larger evangelical denominations had more programs in common, and so things could be evaluated by a set of more “universal” questions.
I found out later that many other people who had been around churches for at least a few decades were equally perturbed by the survey, and for similar reasons. I spoke to the church leadership about the survey and why they were using something that seemed so outmoded. I was told that this evaluation form had indeed been used for something like 25 years at least, and they chose this tool to survey congregational attitudes because it was computerized and so they could easily compile the results and compare them with nationwide results from similar-sized churches. (I don’t know if they chose this on their own, or by recommendation of denominational officials.)
I realize now that the basic technical problem in terms of methodology was this: This tool may once have boasted of high validity and reliability for the church models of the 1970s and ’80s, but the world had changed significantly, as had church models. So, this questionnaire was marginally valid in the first decade of the 2000s. Though it could accurately measure “church satisfaction” of a congregation member 25+ years ago, many of its questions no longer dealt with current approaches to ministry, and therefore to “satisfaction” with church ministries. Although, if your church hadn’t changed at all since Ronald Reagan became president, it would have been both valid and reliable. However, in the post-Clinton era, probably neither.
This experience gave me more evidence reinforcing my belief that the ways we ask our questions precondition our answers, and also of my belief that what is missing is usually what trips us up. This church’s leaders had sincere motives for doing this survey as a way to prepare for consultations on an upcoming pastoral transition, but the choice of instruments was not helpful.]]
Key Presuppositions that Block Culturally Accurate Assessments
I would suggest that we definitely need new tools for new questions being asked at new cultural edges. The best tools we can develop must be based on strong understandings of dynamics in how new cultures are created, and how to match ministry paradigms that will fit contextually in various cultures. But before we can attack that strategic task, we must resolve one of the major presupposition problems ensnaring many church leaders in North American: We have not yet done well to distinguish between “emerging culture” (singular) and “emerging cultures” (plural).
When church leaders of any age or generation speak of “emerging culture” as a singular entity, this usually indicates a traditional or modernist bias, which assumes that there is one coherent “postmodern culture.” The twin assumption of this one-postmodern-culture approach is that, if we can only crack “the postmodern/emerging cultural code,” we could reach all postmoderns in one right program (often with the expectation that a church-planting movement will automatically ensue). Though these presuppositions appreciate contextualization, they do not seem to consider the underlying social network differences between 1st-century paganism and its 21st-century counterpart. With today’s North American cultures being so individualistic and isolative overall, why should we assume that a “people group movement” is the probable (or even preferred) pattern?
This only-one-postmodern-cultural approach helps explain why some “postmodern” Christians are repulsed by what have become The Standard Approaches to postmodern church and ministry. If all postmoderns are alike, then why do some (if not many) postmodern disciples of Jesus roll their eyes to object to the very idea that their values, cultural behaviors, and worldview are more or less perfectly expressed by such authors as Leonard Sweet, Brian McClaren, Sally Morgenthaller, and/or Erwin McManus; or by such parachurch movements as Emergent Village, The Ooze, and/or CityReach? [[Remember, this was 2003 … and there weren’t exactly a whole lot of authors or movements considered to be “postmodern” at that time. When I re-edit this in the future, I shall most certainly at least double or triple the length of the list!]]
The Battle Between Different Definitions of “Success”
If my analysis is even partially accurate, I believe it helps explain why an emphasis on traditional and quantitative assessments of “success” that are expected/demanded by the single-emerging-culture advocates eventually alienate the multiple-emerging-culture advocates. The accountability systems and definitions of success are generally imposed by funders and leaders from the single-emerging-culture view, most of whom are outsiders from emerging cultures; many of those expected to conform to these systems, but find them alien instead of indigenous, are actually insiders in the culture(s) they seek to reach.
I have continually witnessed this problem sink the efforts of my emerging-cultures church-planting friends since the mid-1990s. For instance:
- I have acquaintances who have left pastoral ministry after the breakdown of their attempts at “tweaking” Purpose Driven Church and/or seeker-sensitive models for postmodern or multicultural target audiences.
- Other church planters found a significant level of frustration and disappointment when the postmodern audiences they targeted did not respond to advertising or to church attendance. (But then, I now believe that these pastors were not substantially “postmodern” in their personal culture, in my understanding of that term, so why should they have expected that the very “postmodern” people they sought to reach would be drawn to them? If their “attractional models” failed, perhaps it was simply because they were not the right “magnet” for the people they were drawn to …)
- Several friends have had church plants “go under” or face radical (and potentially fatal) restructuring because denominational program support was limited to two or three years, and the church was expected to prove itself to be “self-supporting” by the end of that period. I doubt such restrictions would ever be imposed on a “new work” in an international mission field. Yet those working among North American postmodern peoples who speak the same language (but not the same lingo) are de-funded. [[So many leaders in denominational hierarchies seem lacking in foresight; do they not understand that their self-supporting church planting systems almost automatically lead to non-sustainability? Either you create a church that “succeeds” by traditional paradigm rules and thus does not connect with emerging culture people who don’t value those dimensions of “success,” or the planters try to carry forward a truly contextual church model that does connect with people of the emerging cultures but takes too long to become “self-supporting” even though it’s on the right path toward becoming sustainable.]]
- Another church planting candidate friend had the odd experience of receiving an average score under 2.0 out of 5.0 from the traditionalist team leader using the Ridley assessment system (based on modernist definitions of church planting skills and success). Meanwhile, the more postmodern-friendly assistant assessor who heard the exact same 200-plus minutes of interviews, gave the candidate a score over 4.5 – based on strong suitability for work in postmodern contexts.
Not all of these individuals have been lost to future church planting, but I could cite many examples of men and women who have grown so disillusioned with standardized church and denominational structures that they refuse to participate any longer in what they consider a toxic relationship of being overpowered. Sadly, some of these have lost their vision for intergenerational ministry, while others are still willing to relate with leaders of existing churches as long as there are no strangling strings attached. Some newer church planters who are aware of this double-bind now refuse to connect with parachurch or denominational funders in the first place, perhaps to the detriment of both sides of this mindset divide.
(Sidenotes: I realize that anecdotal evidence is not the same as quantitative or qualitative research aimed at in-depth studies of a particular phenomenon, in this case, ministry in postmodern cultural contexts. Narrative accounts do not fully reveal problems or solutions, but at the very least they should be credited with raising pertinent issues. And no Christian funder or organization I am aware of has backed any kind of substantial research into emerging cultures – plural – and how that relates to contextualization.) [[That was written in 2003 – does anyone know of any changes in that reality? In the past five years, has any organization funded extensive research into emerging cultures and contextualization?]])
Theory, Theologies, and Methodologies
I’d suggest that the more traditional focus on “effectiveness” or “success” as defined in quantitative terms (e.g., number of contacts and/or converts, attendance, money collected) absolutely does not work with the emerging organic ministry paradigms because most cultures emerging at the edges of postmodernity are inherently anti-formulaic. This means we must cope with an apples-and-oranges problem, with formula-driven modernist apples versus anti-fomula-driven postmodernist oranges. Also, values and ministry paradigms in many traditional/modern contexts (e.g., conversion-and-quick-church growth orientation) are antithetical to those in postmodern cultural contexts (e.g., slow-but-deep discipleship-based growth).
Significant qualitative questions can be asked in order to assess these processes, which do not yield themselves to traditional quantitative analysis. However, to develop such a set of questions requires that we understand why particular ministry processes appear to work or not work among various emerging cultures. We need a theoretical perspective that explains the “why,” so we can wrestle with the “how better to” aspect of connecting with people of such cultures.
Any underlying theory leads naturally to resonant definitions and methodologies of ministry goals, values, purposes, vision, success, etc. There are few comprehensive underlying theories available as yet for work in postmodern (philosophical and deconstructive) and “post-postmodern” (interdisciplinary, integrative, and constructive) cultural milieus. [[Why is that? We’ve had over a decade to consider the cultural shifts of the previous few decades, the unfolding and morphing of ministry from “GenX” to “postmodern” to “emerging” to “missional.” Why have we not integrated our findings well?]]
Most theologian-practitioners are not theoreticians. Most theoreticians are not practitioners. So, it seems to me, what has emerged has been a patchwork of partial theories. For instance, some theories have come from practitioners with passion and expertise in one type of ministry activity (e.g., ancient/future worship and theology). Their theory of ministry governs just that one area but does not necessarily translate well to creating all structures and methodologies in churches. Others who are more theologically oriented often ask inadequate key questions that slant everything that follows in a very particular direction. (For example, I believe the question “What is the mission of God?” inherently leads toward a more modernist understanding, since the use of the singular – THE mission – presumes there is only one thing that is “right.” A more postmodern-friendly exploration comes simply from shifting the question to the plural: “What ARE the missionS of God?”)
When the “GenX/postmodern ministry movement” got rolling in the mid-1990s, the two most frequent questions I heard for the next five years were, “What’s working for you?” and “What does GenX/postmodern ministry look like where you’re at?” Both questions focus on surface methodologies, ultimately divorced from the local culture since the impetus was clearly, “Tell me your tips so I can try ’em out where I’m at.” I believe this sincere but misplaced overfocus on the “whats” of methods and pragmatism, without an underlying theoretical base to explain the “whys” and “wherefores” of appropriate contextualization, will always lead to problems. I think it explains why, in part, we’ve been seeing a number of seeker-sensitive and purpose-driven churches in the San Francisco Bay Area go into confusion, stagnancy, and sometimes complete demise. There was talk of context, but not necessarily substantive research underneath into the “cultural production” of a group, instead of just what they consumed, according to their “felt needs.”
[[In recent months, I’ve used an updated computer analogy to help illustrate this problem of a patchwork model without a cohesive underlying perspective, and its resolution. I first talked about paradigms as the equivalent of computer platforms in the early 2000s, and have periodically updated my metaphor. Here is the most recent version, from Paradigm Profiling in The Missional Zone Part 4B-The Staff-Led Church is Inherently NOT Missional:
Paradigm Shifting in a Computer Analogy. As a reminder, just because each methodological model I critique demonstrates deficiencies or excesses in relation to the missional paradigm, the model can still be changed. Because models flow from the assumptions in their entire paradigm system, it will take far more than attempts to micro-tweak or remix the model with various patches and de-bugging, or by adding some emulator program so the old paradigm PC desktop is disguised to look like a new paradigm Mac. Sluggish functioning won’t keep you going in the world as it is becoming. At some point, getting up to speed will require deleting the temp files and your search history, and defragging to reintegrate all the bits and pieces into a coherent system – and/or make the leap to an entirely different platform, realizing that not all application programs will be able to translate and transfer!
In relation to this article, the overall point of “defragging” or changing operating system/platforms is that as we shift our paradigm, we must also update our assessment tools to match the newer definitions of “success.” That will help shift from pinpointing or measuring where we have been, and refocus our attention to where we are going.]]
Assessment Systems for Apostolic [[Start-Up]] Processes
Emerging cultures may have leftover churches and ministries in their midst. But, the way I see it, they typically require new start-ups, not just repurposed or renewed versions of the old. So – can apostolic/ministry start-up processes in emerging cultures be measured for effectiveness? Yes, I believe they can. But what will it take to be successful at defining multiple types of “success”?
One of two approaches would likely help. The first approach is to develop an understanding of emerging cultures and appropriate discipleship, ministry, and church approaches in those cultures. That informs us for developing assessments of oranges that apply to oranges. To do this, we could get together a group of theoreticians, theologians, and practitioners from within this cultural milieu to work together on describing the processes as they see them. We’ll have to realize in doing this that one key problem is that there is no language at all to describe some of these paradigms, or there is no shared/common language to talk about them. That problem has to be addressed. In addition, I would suggest that the more people who have at least dual roles, the better (e.g., theoretician-theologian-practitioners, theologian-practitioners, etc.).
A general guidebook that may be of help here is Robert Webber’s The Younger Evangelical: Facing the Challenges of the New World. It is more comprehensive in scope than most other attempts at this task of paradigm analysis. And his categories may give a useful framework for organizing information for those who represent ministries in the field. But it may display the problem that almost always appears when we make broad generalizations – the farther away you go from the aggregate big picture to the specific individuals or local ministries, the less the generalizations apply. So this task force must work to describe multiple case studies of ministry within similar categories, and seek to discern patterns as well as the particulars.
A second approach is to work with a theory that could be applied to all churches – in other words, a way to test all fruit, not just apples or oranges specifically. I have been working on this kind of systems approach that can guide assessments of any church, because it focuses on what all congregations should be and do – what I call transcultural ministry or “Kingdom Culture.” If this works, it would still prove valid for whatever major types of fruit show up in the emerging culture edges after apples and oranges have both seen their day.
I have been working toward this “meta-theory” fruit-inspection perspective since the mid-1990s. I found myself consistently dissatisfied with the overfocus on pragmatic methods, formulaic church planting models, and partial theologies and theories that I heard promoted at various events for “GenX/postmodern ministry leaders.” From my own attempts at a comprehensive approach to understanding culture and contextualization, I’ve written papers that critique various “received wisdom” conclusions and assessments, and hope to finish writing about the basic cultural analysis theory by early 2004. [[Update: That part is in fact finished! I’m working now on additional concepts in contextualization and development of countercultural discipleship.]]
We can develop appropriate assessment tools, whichever route we choose to pursue – orange-specific or general fruit – or perhaps with other approaches I haven’t even thought of. I believe most of them will be qualitative to help us discern the depth of our growth as disciples in communities and congregations. Some of them will be quantitative to show specific progress toward reasonable numerical goals and/or timed trajectories. In all cases, we must trust the Spirit of Jesus Christ to help show us the ways …
© Brad Sargent. Version 1, April 2003. Version 2, October 2003. Version 3, August 2008 – minor editing, and a few insertions of new material as marked [[in double square brackets]].
ADD-ON #1: MISSIONALMETRICS AND CHARISMETRICS
On August 12, 2008, Brother Maynard posted Call it Apostolic Fallout. There he dealt with the then-breaking news about the filing for separation from his wife by Todd Bentley of the Lakeland, Florida, revival movement, and what this could mean for those who conducted his “apostolic commissioning” about seven weeks earlier. In light of my above material about validity, reliability, and assessment tools, you may want to read his entire post, which includes the following comment I wrote there on “missionalmetrics” and “charistmetrics” for evaluating success in these two paradigms.
This generates a lot of thoughts, Bro. Maynard …
Success just doesn’t mean what it used to in the old paradigms. Whilst we’re trying to figure out the meaning of “missional” from our unique “post-” contexts (post-evangelical, post-liberal, post-charismatic, etc.) – we’re having to change our definitions and measures of “successful church” to match our new paradigm. In other words, as we change our thinking patterns and methodological models, we also need to adjust our “missionalmetrics.”
It’s been a few decades since my last course in sociological research methods and statistics, but I do seem to recall that two of the key issues are:
- Validity – does the assessment/method actually measure what it says it does?
- Reliability – does the assessment/method get basically the same results over time?
Social researchers want ways to measure things that prove both valid and reliable. Have we got that yet for the missional movement? I don’t think so, and since I have some possibility for creating some specialized missionmetrics, I’ve been working on that for the last few years … maybe a few more, and we’ll see what the beta-testing says about their credibility …
Meanwhile, you suggest in this post that the equivalent to “metrics of success” for everyday Christian life have remain unchanged from the beginning. Tying that together with the social research concepts, could we say that valid and reliable discipleship is demonstrated by an ordinary, non-spectacular (but still quite wondrous) manifestation of The Big Nine – love, joy, peace, patience, kindness, goodness, faithfulness, gentleness, and self-control?
And might I be so bold – or crazy – as to suggest that the metrics of success for extraordinary Signs and Wonders have also remained unchanged from the beginning? With a purpose of demonstrating God’s presence and power, shouldn’t genuine miracles always be verifiable, validated, and the demonstrators prove reliable? If these factors don’t all align – bogus performance and/or bogus performer – then don’t we have both the permission and the obligation to label it a counterfeit, even if something quite spectacular has happened? The Scriptures are clear that God is not the only being with power – though His trumps all others – and that not all miraculous events are from Him.
Call these assessment criteria “charismetrics,” if you will. And if the miracles themselves do not prove themselves credible through charismetrics, and their conveyors do not prove themselves credible through some reasonable degree of disciplemetrics, then I’d suggest we’re free not only to wonder about the “signs,” but assume these don’t add up to success in missionalmetrics.
Nothing all that much new here, but if we’re going to talk about contextualization and being countercultural as part of our journey toward missional, I think it makes sense to consider language that might be used by people in our neighborhoods to measure the so-called “success” of our lives and lifestyles as followers of Jesus Christ. And could it be that the better ongoing measure of missional success is the not-so-miraculous Big Nine All The Time than the occasional S&Ws, even if we find ourselves living in a highly occultural society where manifestations of non-human powers are expected?
ADD-ON #2: COMPLETENESS, INTEGRATION, RELIABILITY
On the same day Bro. Maynard posted on “apostolic fallout,” Len Hjalmarson posted on Marks of the Church. He writes about polarities of DNA, and the importance of keeping them together instead of dividing them asunder. That sparked my thinking on paradoxes, and their importance for maintaining dynamic tensions for holistic ministry. I responded with this comment that lists two more dimensions to a holistic, paradoxical approach to a set of assessments: completeness and integration. Something to think about …
Amazing how the seemingly simple shift from either/or thinking to both/and can put the “dynamic” back into the “tension.”
I’ve been working on a set of inventories that attempt to inventory a healthy system in terms of at least its:
- Completeness (comprehensive range of items or categories)
- Integration (the items are interconnected, not segmented)
- Reliability (proven by testing over time)
It’s a challenge … and yet, maintaining the polarities keeps it much more alive and dynamic, while splitting them makes it easier to lose one part or the other, to the detriment of the system as a living whole.