Responsive web design is such the hot topic these days and it clearly has it’s proponents and detractors. One common complaint about RWD has to do with the size of site and the impact that has on performance. It’s all too common to find responsive sites sending the same amount of information to mobile devices as they do to desktops. Device detection services and techniques can help us optimize our content and serve the right data to the right device.
At this point you amy be saying, “Isn’t one point of RWD to have a single code base and to deliver the same content to everyone?” The answer is yes and at the same time no. Yes, people are often looking for the same content regardless of what they use to view your site. No, they don’t want that 1800 x 1200 pixel full screen background image sent to their phone.
This is where detection techniques can come in to solve for those problems. Adapting content in a way that provides an optimized but not reduced way. We are not taking anything away because someone is using a smaller device. We simply want to optimize what is sent in order to maintain a good performance and in turn good user experience.
Today I’ve asked Chris Abbott, Mobile Detection and Analytics Expert, from DetectRight to answer some questions about the role of device detection, dispel some misconceptions and how device detection can help in developing optimal web experiences.
Q: I’d like to open this by asking if you could explain what device detection is and how it ties into developing for web today.
Device detection is all about knowledge. Knowledge is power. There’s lots of knowledge that a client-side process cannot get, or where it’s too late to use it.
Device Detection provides that knowledge. Something as simple as device type or diagonal screen size, for instance. Or something as complex as knowledge of support for a particular audio codec.
Responsive is naked without Adaptive.
The question then is: “OK, we need adaptive, how do we do that?” That’s where DetectRight come in.
Q: Is the idea that “browser sniffing is bad” an outdated concept left over from when it was actually much more difficult to accurately detect device? Is it time to re-examine what we know and how things have changed to improve device detection?
The phrase “browser sniffing is bad” is mixing a lot of different semi-truths together in a wholly misleading way.
Walking is risky. But you can’t say “walking is bad and you shouldn’t do it”. You look at the risks, judge by your prior experience and available knowledge, and make your own decision. You also look at the alternatives. In a pedestrian area, walking is your only choice.information that’s REALLY needed, you need to use device detection. Which is a lot more than mere “browser sniffing”, by the way.
There is a great lack of data in this area which allows mistaken impressions to grab hold.
“Browser sniffing” is a poor second to feature detection for implementing desktop sites. Remember the days of sites kicking you out because you were missing a crucial string in your user agent? No one wants to return to those days, and people don’t have to.
But this is 2013: and device detection has an additional (and more crucial) task: implying the hardware characteristics from the browser signature and pulling in offline information about it, because there are no better sources available! And that information is objectively useful to a developer.
There are actually millions of sites out there doing device detection: the mobile plugins for CMS like WordPress or Drupal, Yahoo’s YUI framework, and sites which use the commercial device detection systems…
Unfortunately measuring accuracy in this realm is very, very difficult. We’re making good progress in this area, and hope to bring some factual data to the party. YUI, btw, is particularly poor in its detection.
Q: UserAgent misreporting is often cited as, or at least one of, the Achille’s heal of device detection. How bad is this problem? How often do devices “lie” about who they really are based on your experience?
This is a big problem, because as I said, it’s mixing up lots of different truths to form a falsehood.
Just because a user-agent can be faked. That doesn’t mean everyone is doing it.
There are ways of telling it’s been done, and often clues are left behind in the result, as well as additional headers added on that the user couldn’t possibly know about which show the truth.
Just because a user-agent contains useless information does not mean that it is useless.
There’s lots of “accidents of history” in useragents: the whole “compatible;” thing in MSIE strings, for example. Or the “like Gecko” nonsense. People use these examples to deny legitimacy to device detection, and it’s wholly wrong.
First of all, you quickly learn to ignore the rubbish: or rather, to prioritise a detection so that better quality tokens are prioritised over poor quality ones. A case in point: Chrome for iPad useragent has an identifier for Chrome, but it also keeps the one for Safari in. So, any analyser has to prioritise the Chrome token over the Safari one.
In most device detection systems, user-agents aren’t even analysed: they’re simply used as identifiers to match against a database. I think that throwing away data like that is wrong, but that’s one of the reasons why I think my own DetectRight system is superior.
Just because device detection has been done badly doesn’t mean that it’s an invalid concept.
Most device detection historically has been done badly.
For instance, using regular expressions forged in the mists of time and handed down from father to son, and never checking or updating it.
Or, putting too much trust in free open source solutions that rely on the “community” to actually keep a database updated, fresh and consistent (this did not, will not, and cannot ever happen!).
Some companies just don’t update their database frequently enough.
Just because device detection is hard, doesn’t mean there are no accurate solutions.
Yes, it’s hard keeping up with new devices and user-agents. It’s hard writing detection systems. But that doesn’t mean it’s not worth doing, or that everyone should give up. It’s hard writing relational database engines, or anti-fraud measures, or anti-virus engines. Should they be given up?
There are commercial organizations dedicated to the hard work needed to keep these data and systems up to date. And the organizations have to be commercial because it’s not cheap to run cloud systems, or gather data, or pay employees. If the industry had been more sensible about this, then we wouldn’t have this situation.
But we do: the telecommunications industry is making developers pay for its own hubris. Device detection is a hack, but a necessary one. So are media queries. Reality is heuristic and hacky. That’s just how it is, and how it will always be.
One of the rally cries from the responsive community, myself included is “one code base to serve them all.” This is a lofty goal that we’re shooting for here. One HTML file that can be sent to any device regardless size, that’s fully accessible, and is SEO friendly. The problem is that they are usually not very well optimized. How can we use device detection to adapt our markup without sacrificing the good stuff?
Adaptation is not the same as fragmentation, and nothing about device detection stops you having one URL and one underlying codebase. What device detection allows you to do is to assign resources at an earlier stage in the process, using superior knowledge of your customer.
So Device Detection/Adaptive Design (which can work with the best tools in responsive design since they’re at different layers) enables superior site experiences.
I simply don’t know why it’s become so “manly” to insist that all server-side be ditched. It might be the psychological trait where intentionally making things harder for yourself is seen as more “virtuous”.
RWD has tied its own hands, declaring on behalf of web developers everywhere that “adapting thy HTML shall be a mortal sin, only CSS layout changes shall be permitted”.
There are many use cases when it’s desirable to serve different markup to different devices: for instance, optimising your interface and code for a touch screen experience, or inserting a different ad server/analytics routine (the ad servers use device detection even if you don’t: something to think about!).
So it’s all about knowledge: going from zero knowledge about the user to lots of knowledge about the user.
Another important aspect: analytics. With a server-side detection you can collect first-hand analytics in much more detail than with Google analytics. You can even use the results of the server-side detection to feed Google Universal Analytics.
Q: Another aspect of RWD is the idea that we are suppose to be designing “device agnostic” sites. The whole idea of device detection seems to fly in the face of this. What’re your thoughts on how incorporate device detection in what should be a device agnostic workflow?
I would question the whole premise.
A “device agnostic” site is like a “consumer-agnostic” TV program. The concept of “device agnostic” is entirely driven by the needs of the developer, not the consumer.
The structure of a website and the way it’s presented (like the pace, format and editing of a TV show) has to be tailored to the audience and their medium.
The other thing is: how do you define “agnostic”? As soon as you use one media query, you’re caring about the size of a device, and implementing multiple code paths and experiences for the user. As soon as you utter the words “progressive enhancement”, that’s not one codebase: that’s many possible codebases. It’s just where you put them.
Q: CDN’s allow us to hook into code libraries like jQuery ensuring that we always have the most up-to-date version available. As a fail-safe we often save a copy to our server in case the CDN goes down. What sort of fail-safe is possible when using a device detection service?
If you’re serious about your website’s performance and reliability, don’t call out to third-party web services from the server.
All good device detection services have local deployments available, and in fact that’s the preferred method of use: there are all sorts available, though most are concentrated on PHP, Java, .NET and C++. Some device detection systems plug into Varnish Cache or Apache.
Anyone designing a web service for device detection will have put a great deal of thought into making it as bulletproof as possible, including geographical redundancy, load balancing, and multiple servers. If something like Amazon goes down (as it has), then the Internet probably has bigger problems than its device detection.
Any local solution will have been tested in the field and will have at least some satisfied customers. These days, it takes a lot to bring down a device detection solution.
Q: Is a server-side solution always the best choice? Why might someone choose a client-side solutions instead?
By the time you’re client-side, the important decisions have already been made, and you’re at the mercy of the user’s device and its connection. It’s already disconcerting to see a site take 10 seconds to load messily then adjust itself to look pretty (a really annoying side-effect of relying on a device’s CPU). To load a site, and then change the device type on the fly… well, that’s messy.
But it’s possible to run a device detection system client-side. We’ve got a client which does that asynchronously from a cloud server.
Q: Tell me about DetectRight.
DetectRight has been around since 2006.
The DetectRight system is a device detection library you can deploy in PHP, Java or .NET. It has thousands of possible device data points, four times as many devices as anyone else (this matters!), much more accurate detection than media queries, regular expressions or any other detection system, and is also the most cost-effective and friendly company in our sector.
As a company, we genuinely care about the small business end of the market. We sympathise with the pressures developers are under, the shapes of the deals they do and how that’s impacted by the need for third party technology, or the pressures put on solution providers by inadequate budgeting.
That’s why we love talking about device detection and how it can help: and in coming up with innovative ways in which we can help others improve the mobile web.
There’s information that only we can see that needs a wider audience: such as details of how accurate device detection is across systems and CMS, how many user-agents are faked, what percentage of devices are cloaked… etc, and working on the killer question: “how does using device detection affect the bottom line”?
We are also pioneering work on important questions such as: “is this a real mobile device?” which might have some bearing on fraud or bot detection.
We also want to share our hard-won knowledge and systems with the ecosystem: this interviews is a start.
We’re also planning a device detection social network that developers can join for free to get access to device detection advice, standalone and cloud systems and resources, and can contribute back according to their ability to do so.
The two-way sharing aspect of this is important, which is why dialogue with developers and personal relationships is crucial.
Q: Do you think that Responsive Web Design has lived up to its promises to developers?
It allows developers to think they’ve succeeded, but often at the expense of web visitors who experience slow loading, greater load put on their device and connection (that they’re paying for), disconcerting rearrangement behaviour, and other quirks. It’s still better than a “pinch and zoom” website, of course. There was a time when the commonly held belief was that all devices would support full websites and so no device detection was needed.
What we’re selling, is mixed news: “no, everything’s really screwed up, but it’s ok, we’re here!”.
That’s a difficult sell because the human instinct is to believe everything will just work out.
But if you can get over that instinct and enter a reality-based world (with a little help to understand how vast that reality is), it gives you an inherent competitive advantage over everyone else. And that has to improve the bottom line, as well as the mobile web generally.
I’d like to thank Chris for taking the time to share his thoughts on the subject of device detection. I hope you, the readers, will share your thoughts and any experiences you may have to further the conversation.