Does the European-based camera and lens testing lab DxOMark have some kind of vendetta against Canon? Or does the world’s leading camera brand have a serious and growing technology deficit?
DxOMark is the ‘go-to’ site for those interested in digging into the hard data on comparative performance of camera sensors and optics – the two components upon which camera performance are largely based. While there are plenty of websites reviewing cameras and giving them star ratings, gold, silver and bronze awards and model of the year trophies, DxO Mark claims its tests are more objective. It has, according to its website, been putting ‘camera sensors and lenses through rigorous objective hardware testing, using industry-grade laboratory tools to establish the most comprehensive reference database from thousands of camera and lens test results since 2003.’
Although DxO claims all top ten digital camera and smartphone manufacturers are customers of its DxO Analyzer camera image quality benchmarking system, Canon and Apple are not listed by logo as customers So something is amiss there.
DxO also develops image processing software for photographers and last year launched its own camera – the DxO One. (Which sits at a fairly humble number 36 in the DxOMark ratings, in case you were wondering.)
– So it’s passing curious, and a subject of occasional comment on various photo industry forums, that there are no Canon cameras in DxO’s top 10 ranked models. And only one in its top 20! It would be natural to assume that some of Canon’s big guns, such as the 50-megapixel 5DS would rate a top 5 position at least. It’s 29th. Or the most expensive pro DSLR in the line up, the 1DX MkII ($9300). It’s 23rd.
How can the sensor in the Sony Cyber-shot DSC-RX1, a 24-megapixel premium compact released in 2012, have a better ranking than every Canon camera, including the top-ranked Canon to date, 2016’s EOS 5D Mark IV? The Sony, with a Carl Zeiss Sonnar T 35mm F2 prime lens, lens retails at around $700 less than the body-only Canon. While sensor performance is not the only thing, it certainly is, in the modern parlance, a thing.
Canon only has five models in DxO’s Top 50. By comparison, Nikon has 20 cameras and Sony has 13.
And speaking of lenses, why is it that the top ranked Canon lens, the Canon EF 35mm f1.4 II USM (released last year) is only number 37 overall on the DxO lens hit parade? But we will focus down on sensors in this article, or it will all get too unweildy.
DxOMark tests sensors on three criteria: ISO sensitivity (speed); noise (standard deviation, signal-to-noise ratio, dynamic range, and tonal range); and colour sensitivity. There are a whole range of other parameters – AF speed, continuous burst speed, build quality, user interface, etc, which it doesn’t look at, and other camera review sites do look at. But all things being equal, the better the sensor performs on those three critical measures, the better the image quality will be.
It has to be said that, if DxO is right, Canon must do a fantastic job on all those other parameters to maintain its position as market leader, and retain the loyalty of thousands of pro and advanced amateur photographers around the world.
Across the counter
So if DxO is right about the performance of Canon sensors – and lenses for that matter – does this have implications at the photo retailing sales counter? How do staff respond to a clued-up customer who challenges a recommendation for a Canon model based on the DxO results? Canon gets the heebie-jeebies about retailers ‘switching’ customers to other brands, which seems to be part of the rationale for it moving more aggressively into the ‘retail space’. But hey, maybe that’s exactly what retailers should be doing if the key relationship is between the store and the customer, rather than the store and the camera company. (Just sayin’!)
Or maybe all current digital cameras and lenses are so good that all this pixel-peeping is irrelevant nonsense. As far as PhotoCounter is aware, Canon has never made any public statements defending its sensor and lens technology against the assessments of DxOMark, so we just don’t know.
And how about all those camera reviews, which according to the current market research are so influential in buying decisions? What’s going on there? How come they don’t reflect the DxOMark results more closely?
This DxOMark versus Canon thing is maybe a leading indicator of a business in technological decline. But understandably it’s not something camera reviewers seem to want to buy into. It’s not something the photo industry establishment really wants to buy into.
Google ‘DxOMark’ and ‘Canon’ and there haven’t been many open discussions on the subject. Usually it turns into an unseemly insult-athon between Canon fanbois and the rest of the world. Perhaps the most useful and even-tempered discussion was last year in the professional-leaning Image Resource website, following a short video entitled DxOMark sensor scores – 10 things you didn’t know.
The video made a number of points in partial defence of Canon, noting among other things that there were a whole bunch of critical criteria around general camera performance that DxOMark didn’t test. Which is correct, but as some readers pointed out, sort of irrelevant if sensor performance is what we are looking at.
One reader contributed a short essay in response, noting sensors are an expensive part of DSLR construction and, in his opinion, Canon has saved money by not improving its sensors. Another said he found the video useful, but also found the DxOMark ratings actually correlated to his real life experience using a number of different cameras.
Dynamic range appears to be the consistent criticism of Canon sensors, according to DxO and general industry feedback, with the latest iteration of the Canon EOS 5D making up some ground on its competitors.
But Sony, which manufacturers its own sensors and supplies chips for Nikon, Pentax and Olympus among others, representing around 40 percent of the entire market for advanced camera sensors, is a fast-moving target. It has massive economies of scale compared to Canon and, according to its own boffins, a five-year lead on the competition. It would take a considerable investment for Canon to catch up. But why should they, in these straitened times for camera manufacturers? They have managed to maintain market leadership with their existing sensor technology: ‘…they don’t have to put a better sensor in their cameras. People are still buying them and they are making a lot of money. When customers get irritated at the older technology being sold in new cameras, Canon can just pop better sensors in for the next release cycle.’
But the 64-megapixel question is – where are they going to source that better sensor?
When it comes to lenses, Canon fares even worse at the hands of DxOMark. Here Sigma, Tamron and even newcomers like Samyang are putting out highly rated Canon-mount optics at a fraction of the cost of the ‘original’. Where once photographers were locked into paying monopoly prices for top quality optics to match their Canon (or Nikon or Sony) bodies, that’s no longer the case. And so those handsome profits from lenses, which could be used to support further technological development, will inevitably be pared back – either through lost sales or in response to price competition.
So – is it a serious and growing technology deficit we are looking at, or as one DPReview commenter asked, ‘does DxOMark flat out hate Canon?’
– Keith Shipton