Companies are the stewards of the community NOT the manager . . . it was something that was deeply ingrained into every single employee’s psyche at eBay. (eventhough it may not seem like it from the outside ) Apparently that lesson is being learned everywhere these days : youTube, eachnet, myspace, digg . . . the latest is that of facebook.

There are a lot of random posts on the controversy but only two actually offered some solutions and suggestions from the company/product management angle : Ed Sim & Ken Norton . . .

I wrote about it before on so called Web 2.0 Community Management but I think Stewardship is probably a better word for it. . .

A/B testing of new features is actually really really hard (like Ken said). You have a few options (not mutually exclusive)

1) Blind A/B
2) Opt-Out A/B
3) Opt-In A/B

None of them are perfect and in the end the community might still scream and yell . . . .

1) Blind A/B
This just doesnt really work for community centric businesses . . . peoeple are just so attached and familiar to the site you need to be extremely transparent, otherwise you are going to get support calls (people think the site is broken) or another riot on your hand. Google/Yahoo/Amazon can do this because they have huge amount of “blind” traffic coming through the site. . . Google especially is already a black box so people dont expect it to be transparent in the first place. For eBay and, my guess, social networking sites; it will not work.

2) Opt-Out A/B
This is a little better in preventing a riot but you better make sure the A(control group) portion is siginifcantly bigger. Even so, you are going to get huge self selection issue because people will notice the “opt out message” and act very differently (such as clicking around a lot more and become a lot less task oriented). The portion that got sent there by default might still cause a rukus because they believe these features have a huge sunk cost so they will be released eventually anyways - perhaps with tweaks - and you know what. . . he/she is probably right . . .

3) Opt-In A/B
This will get you more input and participation from the community but in the end, the results are not going to be high fidelity. . . or even directionally correct. Those who opt-in either are early adoptors, have an ax to grind, or explorers. Furthermore, whatever metric you are testing for, selection bias will make it almost irrelevant. In many ways, this is the ONLY option community based sites have to be partipatory in releasing new features . . . another thing to do to reduce bias is to run the A/B for LONG time so that you at the very least filter out the riff-raffs who are digging around for dirt (reporters?) and see if your community either adopts its slowly but surely or segments significantly. (or never at all!).

For companies without conversion metrics (like facebook) the above tests are even harder . . . you can survey people at the end of the new experience but expressed and behaviorial feedback is very different. Furthermore, not until they have explore the majority of their OWN use cases for the site, their opinion might change if the feature is rollout permenantly. Retention/repeat visit/page view metrics are not linearly correlated to “satisfaction”, thus the effect of these features might be delayed or compounded negatively by some other factor down the road. And ofcourse relying soley on “feedback” creates self-selection bias on top of self-selection bias.

For companies with conversion metrics (eBay) the sampling bias is a huge issue for opt-in/opt-on. Furthermore what do you do if 5% of the population is hugely against the new feature but the silent majority (based on metrics) seems to be inconclusive or positive?

Llastly, as Adam Nash likes to say to me. . . what IF your most vocal community is the dying majority (as in The Innovator’s Dilemma - also brilliantly point out by Ken Norton). What IF you end up beholden to a segment of the customer base indefinitely and if that segment for some reason slowly shrinks? You do not have the strategic alternative of serving a new segment and grow for the fear of losing your current customer/user base. Many industry giants have died off this way . . .

(Side note, if facebook rolls back this feed feature, some startup should build a social network purely on the idea of broadcasted status . . . they will attract/self select those who dont mind and they will also know that the incumbents will not encroach. And as I suspect, as the web matures evenmore, the who idea of realtime updates will become more common place/desirable).

There are 20+ variables to evaluate when trying to roll out a new feature smartly to a community of users, there is no right answer for all cases, this is what makes this whole web 2.0 thing harder than people think . . .

(Guest Column: Peter Daley, Digital Media Analyst, Rutberg & Co)

During the one month period from July 1 through July 31, 34 private digital media companies announced $403.1 million in venture capital financings. Investments were primarily in the Advertising Infrastructure, Applications, and Content Publishers sectors. There were also 24 M&A transactions announced during the period, including Motorola’s acquisition of Broadbus, AMD’s acquisition of ATI, and Planar System’s acquisition of Clarity Visual Systems.

In this newsletter, rather than our traditional review of the previous month’s news items, we discuss the venture capital, M&A, and IPO activity in digital media over the past two quarters and compare these findings to the activity of 2005.

Venture capital activity in digital media is, by all accounts, robust. During the first six months of 2006, private digital media companies raised $2.4 billion of venture capital through 278 financings. This activity nearly matches that of the full year 2005, in which $2.6 billion was raised through 313 financings. Notably, 245 venture capital firms announced two or more digital media investments during the eighteen month period between January, 2005, and June, 2006.

The robust activity indicated by the financing totals is consistent with our conversations with entrepreneurs and investors. On an anecdotal basis, for example, we note the formation by numerous venture firms of digital media-specific strategies, practices, and portfolios over the past two years. Although raising capital is never “easy,” we believe the current environment is healthy for private digital media companies.

The M&A market remained steady in 1H06 with 181 transactions announced during the period, as compared to 183 and 185 in 1H05 and 2H05, respectively. Further, the IPO market for digital media companies remains muted. During the first half of 2006, only 6 IPOs were filed and 6 were priced in the industry, as compared to 5 filed and 19 priced in 2005. We do not find visibility toward a substantial change in the dynamics of the IPO market in the near-term. As such, we believe M&A remains the far more likely exit scenario for private equity investors.

Overall, we note that venture capital activity has increased substantially over the previous 18 months, while M&A activity has remained steady over this same period. Several venture investors in our conversations have referred to this situation as a “bubble without an exit,” indicating a concern and frustration over the high investment levels but low exit numbers. As we discuss below, we believe that this situation may be, at least partly, due to the early stage of digital media as an industry, and that there are underlying, positive indicators for future M&A activity in the industry.

Venture capital activity in digital media is, by all accounts, robust. During the first six months of 2006, private digital media companies raised $2.4 billion of venture capital through 278 financings. This activity nearly matches that of the full year 2005, in which $2.6 billion was raised through 313 financings.

On a quarterly basis, total venture capital raised during 2Q06 was $1.3 billion, representing 127% growth from the total during 2Q05.

Heading into 3Q06, venture investors appear poised to continue their strong pace. The $403 million of venture capital raised in July 2006 is on par with the $398 million monthly average over the previous six months.

The robust activity indicated by the financing totals is consistent with our conversations with entrepreneurs and investors. On an anecdotal basis, for example, we note the formation by numerous venture firms of digital media-specific strategies, practices, and portfolios over the past two years. Additionally, we hear comments from several entrepreneurs on strong interest by venture firms and on relatively attractive transaction valuations. Although raising capital is never “easy,” we believe the current environment is healthy for private digital media companies.

Size of Financings

In terms of financing size, the average size of digital media venture transactions has remained stable since 2005. During 1H06, the average financing size was $10.6 million, as compared to $9.1 million and $11.5 million during 1H05 and 2H05, respectively.

Interestingly, over this period, transactions have trended toward smaller (less than $5MM) and larger (greater than $20MM) financings. The percentage of mid-sized (between $5MM and $20MM) financings has commensurately decreased. We believe this change in distribution is driven by two factors: 1) increased venture capital for private companies in the Applications sector, which is at an early stage of development and which typically requires less total capital commitments per company than the broader digital media industry; and 2) the emergence of private companies with late stage revenue levels in several digital media sectors, such as Semiconductors, Content Service Providers & Retailers, and Advertising Infrastructure.

Most Active Sectors

The sectors with the largest total venture capital raised during 1H06 were Semiconductors, Content Service Providers & Retailers, and Advertising Infrastructure. Notable financings during the period were: $150 million by Amp’d Mobile, $49.2 million by Digi TV Plus, $48.5 million by MovieBeam, $48 million by AdKnowledge, and $46.6 million by Sling Media.

By number of financings, the Applications, Advertising Infrastructure, and Provisioning & Delivery Infrastructure sectors were the most active during 1H06. In particular, the Applications sector included 46 financings during 1H06, as compared to just 9 financings during 1H05. The significant rise in venture activity in Applications companies is driven, among other factors, by the investment theses in user-generated content and other personal media.

Overall, as per our comments in recent newsletters, we remain bullish on investment theses related to user-generated content, casual video games, and content delivery networks, among other areas.

Most Active Investors

The investors with the greatest number of digital media investments in 1H06 were Intel Capital, Sequoia Capital, Cisco Systems, and Draper Fisher Jurvetson. Notably, 245 venture capital firms announced two or more digital media investments during the eighteen month period between January, 2005, and June, 2006. In our view, this demonstrates the broad participation and interest in the industry by venture firms.

Number of M&A Transactions

The M&A market remained steady in 1H06 with 181 transactions announced during the period, as compared to 183 and 185 in 1H05 and 2H05, respectively. Notable recent transactions involving venture-funded private companies include Microsoft’s acquisition of Massive, VeriSign’s acquisition of m-Qube, and Viacom/MTV’s acquisition of Atom Entertainment.

The North America region saw the largest number of transactions during 1H06, with 94 M&A announcements. M&A activity in the Asia region, however, increased significantly, representing 19% of transactions in 1H06, as compared to 8% in 2005.

Most Active Sectors

The 1H06 M&A transactions were primarily in the Content Service Providers & Retailers, Advertising Infrastructure, and Content Creators sectors. The Retailing Infrastructure sector experienced the sharpest rise in M&A activity, with 14 transactions announced in 1H06, as compared to 4 in 1H05. Further, the steepest decline in M&A activity occurred in the Content Publishers sector, with 19 transactions announced in 1H06, as compared to 36 in 1H05.

The IPO market for digital media companies remains muted. During the first half of 2006, only 6 IPOs were filed and 6 were priced in the industry, as compared to 5 filed and 19 priced in 2005. In our conversations with private company board members and CEOs, we find several inhibitors for IPO activity, including the generally unreceptive public markets and Sarbanes-Oxley compliance costs and risks. We do not find visibility toward a substantial change in the dynamics of the IPO market in the near-term. As such, we believe M&A remains the far more likely exit scenario for private equity investors.

Overall, we note that venture capital activity has increased substantially over the previous 18 months, while M&A activity has remained steady over this same period. Several venture investors in our conversations have referred to this situation as a “bubble without an exit,” indicating a concern and frustration over the high investment levels but low exit numbers. In our view, this situation may be, at least partly, due to the early stage of digital media as an industry. Industry drivers of broadband penetration and alternative media business models are just now emerging as visible, mass market forces, and revenues from the associated services and technologies are only recently beginning to be explored, defined, or built. Further, we find that interest on behalf of potential acquirers exists, even if only on an exploratory basis. Although it is infeasible to predict timing or to provide certainty, we believe the underlying dynamics are positive indicators for future M&A activity in the industry.