“Features” has now become a useless concept when evaluating IT projects

The ris­ing scarcity of atten­tion makes the con­cept of “fea­tures” increas­ingly irrel­e­vant. IT and Busi­ness Exec­u­tives need to unlearn using this con­cept as an eval­u­a­tion tool for IT appli­ca­tions. If ven­dors want to increase their mar­ket share, they also need to make sure “fea­tures” is not the focal point of their sales pitch.

Ini­tially, we iden­ti­fied a fun­da­men­tal “Macro” Prin­ci­ple: atten­tion scarcity is deeply reshap­ing busi­nesses. To be action­able, this fun­da­men­tal prin­ci­ple was trans­lated into a strate­gic one: the use of Return on Atten­tion (ROA) as a key met­ric within orga­ni­za­tions. I am now look­ing at the appli­ca­tion of this strate­gic prin­ci­ple to Enter­prise IT, one of the crit­i­cal areas that need to apply ROA much more rigorously.

Return on Atten­tion as a met­ric to eval­u­ate IT Projects

This met­ric needs to be used across all activ­i­ties, but it’s nowhere as impor­tant as in eval­u­at­ing the per­for­mance of cur­rent and future IT applications.

An IT appli­ca­tion is indeed mainly con­sum­ing atten­tion for its users. Infor­ma­tion Tech­nolo­gies are also the domain where the oppor­tu­ni­ties to dras­ti­cally improve ROA are the most preva­lent. Depend­ing on the appli­ca­tion, you can achieve the same activ­ity while con­sum­ing very dif­fer­ent amounts of your atten­tion. The vari­ance is huge. Still, an IT sys­tem is gen­er­ally viewed as a set of features.

How do you decide if a prod­uct is good? How do you choose between two dif­fer­ent but rel­a­tively sim­i­lar offer­ings? Too often, the main cri­te­ria is the func­tion­al­i­ties pro­vided. A Learn­ing Man­age­ment Sys­tem can inte­grate and man­age so many courses, use x types of media; an infra­struc­ture can pro­vide mobile email func­tion­al­i­ties or not; a blog plat­form can embed x types of media, imple­ment track­backs or not, etc.

Unfor­tu­nately, fea­tures cap­ture only a tiny frac­tion of the value pro­vided to end users. In order to cap­ture the full expe­ri­ence, you need to take into account:

• the usabil­ity per­for­mance: what are the results of usabil­ity tests?
• mix of fea­tures: not too sim­ple, avoid fea­ture creep. Ulti­mately, this needs to fit with the cul­ture and work­flows in place.
• adap­ta­tion of exist­ing and new work­flows and processes (this can be man­dated by the tool itself of course).
• the “sexy­ness” of the user inter­face (bench­mark is the con­sumer mar­ket): will your employ­ees enjoy using the soft­ware? If not, unless they have to, they won’t use it.
• the train­ing needs: most non-business spe­cific appli­ca­tions should not need more than 20 mins of train­ing for the new gen­er­a­tion of employees.
• poten­tial to develop increas­ing returns dynam­ics (via net­work effects for example).

All these cri­te­ria (and the list is not exhaus­tive) need to be com­bined to eval­u­ate what will be (or what is) the value pro­vided to your employ­ees, and how much the appli­ca­tion is improv­ing their pro­duc­tiv­ity. The aggre­ga­tion of these cri­te­ria in a sin­gle con­cept can be thought of as “flu­id­ity”, “flow”, but is best syn­the­sized as the Return on Atten­tion provided.

Exam­ples

A sim­ple exam­ple would be blogs in large com­pa­nies. Pro­vide them through Share­Point or Word­press Mu, what you pro­vide is still blogs. When eval­u­at­ing the tools, the basic fea­tures of blog­ging will be iden­ti­fied in both applications—assume here that only the basic fea­tures pro­vided by Share­Point are needed. In most orga­ni­za­tions, the solu­tions will be seen as inter­change­able, based on the fea­ture set. Look­ing at usabil­ity though, and by merely using the two to cre­ate a few posts, you imme­di­ately real­ize that post­ing the same con­tent in Share­Point will take at least twice the time (and prob­a­bly twice the clicks) as post­ing in Word­press. Man­ag­ing the posts, read­ing them, and vir­tu­ally all other tasks sports the same effi­ciency dif­fer­ence. And yet, the fea­ture sets will be seen as iden­ti­cal. Do you think the ROA will be as well?

ROA needs to be used when com­par­ing prod­ucts 1 to 1, but it must also be used for all other strate­gic IT deci­sions. Not con­vinced? Let’s look at mobile email capa­bil­i­ties. The choices you make in terms of infra­struc­ture design (as well as sourc­ing and other domains) will con­strain your options in terms of mobile email deliv­er­ies. Your employ­ees using Win­dows Mobile devices because of your MSFT infra­struc­ture will know first-hand the loss of pro­duc­tiv­ity com­pared to other devices, most notably the iPhone and Blackberries.

It is worth stop­ping shortly on this last exam­ple. How many IT exec­u­tives are eval­u­at­ing the poten­tial plat­forms to be used by employ­ees to man­age email on the go? How many busi­ness exec­u­tives are tak­ing a strong stance and forc­ing their CIO to con­sider these issues? In my expe­ri­ence, close to none, the plat­forms avail­able are just a con­se­quences of other tech­ni­cal deci­sions. Yet, how valu­able is it, for your com­pany, that your employ­ees can man­age emails eas­ily on the go? How much value is there by cut­ting in half (or dou­bling) the time it takes to write an email? Right, the value in terms of pro­duc­tiv­ity is typ­i­cally huge. The direct hit at your pro­duc­tiv­ity will just be the con­se­quence of focus­ing on fea­tures and ignor­ing the global Return on Atten­tion you total infra­struc­ture will end up pro­vid­ing your employ­ees with.

Use ROA as an aggre­gated set of criteria

When com­par­ing dif­fer­ent offer­ings for the same project, or when opti­miz­ing your appli­ca­tions port­fo­lio, ROA is a met­ric that aggre­gates all the sin­gle trade-offs each appli­ca­tion incor­po­rates. You should com­pare prod­ucts, or even projects based on ROA. You can use a mul­ti­fac­eted list of cri­te­ria to eval­u­ate indi­vid­u­ally, then cre­ate a weighted aver­age. Cri­te­ria are split between the prop­er­ties of the appli­ca­tion itself and how these prop­er­ties weave together with your cor­po­rate fab­ric. ROA can indeed be eval­u­ated only with regard to the spe­cific activ­i­ties and actors the appli­ca­tion is sup­posed to enable. A generic check­list can be estab­lished, but it will need to be adapted to each busi­ness process.

An ROA check­list con­tains items such as:

• how much train­ing does the appli­ca­tion require? Is it intu­itive enough to skip a train­ing phase altogether?
• is there increas­ing returns dynam­ics embed­ded in the design? If so, what is their level?
• how many clicks are required for all core tasks?
• how much load­ing time is there for all core tasks?
• are RSS feeds orig­i­nat­ing for core items? all items? no feeds at all?
• how does the appli­ca­tion work on mobile access?
• how does it improve cur­rent work­flows? how does it make new, more effi­cient work­flows possible?
• what are the stan­dards used? are they open? will they be easy to work with to spread data and notifications?

My last point is inter­est­ing in itself: con­sid­er­ing the stan­dards, it actu­ally eval­u­ates the ROA, not only through the end-users, but pri­mar­ily through the IT func­tion. A prod­uct with open stan­dards will increase the ROA the end-users get from it through increas­ing the ROA reaped by the IT func­tion with select­ing this prod­uct. It’s not the point of this post, but remem­ber that ROA needs to be eval­u­ated for all actors in a system.

You have to build a com­plete list and adapt it to both your orga­ni­za­tion and the project eval­u­ated. If you use poorly usable, not sexy, slow enter­prise soft­ware but the man­ual work­flows tar­geted are, as a result of the project, auto­mated, no mat­ter how poor the other cri­te­ria rates, the improve­ments at the work­flow level will result in a very high ROA. But eval­u­at­ing in terms of ROA can make you con­sider alter­na­tives that you would not have evaluated.

How to put ROA into action?

The first take-away is the real­iza­tion that ROA is actu­ally what soft­wares com­pete on, and that the vari­ance is huge. If you look at the con­sumer web mar­ket, all web­sites have to com­pete to pro­vide the high­est ROA for their users. The high­est ROA pro­vided selects the mar­ket leader in an area. In fact, the more you move towards the con­sumer web pat­terns of design, the more your appli­ca­tion is likely to pro­vide a high ROA. Brad Burn­ham, of Union Square Ven­tures, among oth­ers has char­ac­ter­ized this very well:

The folks that built enter­prise soft­ware were vaguely aware that their sys­tems had to be acces­si­ble to the humans that used them but they had a huge advan­tage. The peo­ple who used them did so as part of their job, they were trained to use them and fired if they could not fig­ure them out.

Today, no one tells you to use Face­book. There are no employer spon­sored train­ing ses­sions on the use of del.icio.us. The bur­den is on the designer of the sys­tem to meet a need, enter­tain, or inform their users. They also have to seduce those users, hid­ing com­plex­ity, reveal­ing one layer at time, always entic­ing, never intim­i­dat­ing, until the user one day finds they are inti­mately famil­iar with power and the plea­sures of the service.

It will hap­pen in the same way within orga­ni­za­tions. I have first-hand expe­ri­ence of teams not happy with Share­Point using Base­camp. The haz­ards of such sit­u­a­tions are clear, but more rules or con­trol won’t solve them, bet­ter ROA through cor­po­rate tools will.

How to use ROA for your orga­ni­za­tion? Here are a few leads:

When com­par­ing alter­na­tive offer­ings, do spend 15 mins to under­stand the fea­ture set, but stop after 15 mins and switch to actu­ally using the prod­uct as an end-user and expe­ri­enc­ing its usabil­ity level. Speak­ing to a sales per­son for more than 15 mins with­out exper­i­ment­ing with a demo is a waste of time. This is valid for IT and busi­ness exec­u­tives. for your next sales pitch of 1h30, ask for a short pre­sen­ta­tion and demo after 15 mins. You will then focus on the right questions.

Make sure your part­ners within the orga­ni­za­tion have all com­pleted at least one core task in each of the soft­wares con­sid­ered. Make them do it within a meet­ing. Even with sev­eral prod­ucts con­sid­ered, you won’t spend more than 2 hours to do it. At the end of the meet­ing, you will have elim­i­nated at least a third of them.

Favor appli­ca­tions that have their roots in the con­sumer mar­ket: they already pro­vide a high ROA to con­sumers. While the cri­te­ria rel­a­tive to their fit in your orga­ni­za­tion will need to be proofed, those related to their intrin­sic per­for­mance will already be at good lev­els. Think about your recent 23 years old employ­ees, and their first day in the com­pany. They are either forced to use Out­look, which they likely never heard of before, or Gmail (through Google Apps for enter­prise), which they likely use at home. WHich appli­ca­tion will pro­vide them with a higher ROA?

Incor­po­rate ROA in your processes: make ROA a sec­tion on your risk iden­ti­fi­ca­tion check­list, include it in reviews, etc. Build a mas­ter tem­plate that needs to be used, with cri­te­rion adapted to your orga­ni­za­tion, and require min­i­mum scores.

Con­sider the end to end user expe­ri­ence, and focus on the areas with the most risks for ROA, not for the tech­nolo­gies. Mobile email is an exam­ple of a project part with low tech­ni­cal risks but high ROA risks if not done well. If you are a busi­ness exec­u­tive, insist on walk­ing through the com­plete work­flows con­sid­ered with your IT counterparts.

How do you think today’s orga­ni­za­tions han­dle the use of ROA as a met­ric? Do you see the advan­tages in using it?

Photo credit: Domi­brez