Working together for standards The Web Standards Project

Just over a month ago, WaSP announced the Acid3 Browser Test. At the time, we knew that some of the browser development teams were aggressively working to pass the test. What we didn’t know was how long it would take for a browser to pass the test.

“When we released Acid2, the first browser passed it in about a fortnight. Acid3 is orders of magnitude more complicated. I really didn’t expect to see passing browsers this side of August, let alone within a month,” wrote Ian Hickson, the architect of the test, on his blog.

Well, now we know: It took under a month for not one but two browsers to pass the test. On March 26, the WebKit team announced that their public build produced a 100/100 result; Opera announced that they had an internal build which passed on the same day and they released the build on March 28. Congratulations to both teams!

In the past month we’ve received plenty of feedback and questions about the test. Thanks to the help of readers and browser developers, some bugs were identified and resolved. To learn more about the changes that were made as a result, read Ian’s post which lists the major changes.

Your Replies

#1 On April 7th, 2008 10:30 am liorean replied:

Just worth pointing out: Passing the 100/100 score does not mean the entire test is passed, unless it also passes the smoothness/performance test. Neither Opera not WebKit do that at the moment.

#2 On April 7th, 2008 10:47 am Simon Jessey replied:

As I understand it, the teams have mostly concentrated on trying to get their browsers to pass the test, rather than the more laudable goal of improving their standards support to make passing the test a by-product. Unfortunately, this reduces the effect of passing the test to that of a mere party trick. I would prefer to see efforts spent on creating and passing proper test cases – more like the first Acid test, in fact.

#3 On April 7th, 2008 11:34 am Vectorpedia(Rick) replied:

Congratulations to both the Opera and Webkit team for their dedicated work on this project. Its hard to believe they completed this task within such a short period of time. I would like to see the Safari team make the same progress.

#4 On April 7th, 2008 11:48 am Dušan Smolnikar replied:

From what I’ve read, the Opera build that first passed ACID3 was so skimmed down it had security issues as well as missing features. Have we really come to this? Is passing a “silly test” (sorry for the expression) really more important than security? Sure, who cares if the user’s machine gets a virus or a trojan, as long as the test is passed!

Don’t get me wrong, I do support the effort that has been made to provide the test as well as to pass it, but I’m afraid it was just a race of who passes it first, rather than helping developers build a true standards compliant browser.

To someone with a bit more information and knowledge – did all the work that was put into passing ACID3 (for both, Safari and Opera) really help them become a better browser as far as rendering goes, or were just these specific cases covered?

#5 On April 7th, 2008 11:49 am Chris replied:

Didn’t Eric Meyer say the test was largely bunk?

#6 On April 7th, 2008 2:17 pm liorean replied:

Simon: As for Opera they have not merely passed the test by fixing just the specific issues the test addresses – they’ve got pretty good support for all the technologies used in the test (a few notable new ones that are post-Kestrel additions however), for the most part they’ve been fixing bugs or given additional priority to fixes that were already planned, like CSS RGBA/HSLA colour support.
As for WebKit, their SMIL support is pretty bad, just enough to pass the test, so in that case there is some question of fixing it just to pass the test. That does not mean that WebKit devs aren’t sincere in their want to support SMIL fully however, just that they happen to have started with the parts that were necessary for this test. (AIUI, they’ve had approximately that level of SMIL support in the engine but disabled because it was considered not-fit-for-release yet.)

Dušan: The public Opera build that passes the test uses a variation on the interface that Opera use internally for testing and development of the Core components, GOGI (Generic Opera Graphics Interface). This interface does not have the security features of the desktop platform that Opera uses because it doesn’t need them – it’s purpose is testing the Opera core platform itself. The build is perfectly fine for proving that they have solved the rendering engine issues, however.

#7 On April 7th, 2008 2:29 pm WaSP Member ccasciano replied:

Chris (#5): The test is *a* test, not *the* test. Like the Acid2 test there’s certainly a danger in it getting too much focus and being looked at as too great a barometer of a browser’s capabilities.

Eric’s correct in many ways when he makes a case that Acid3 is coming at things a little differently and perhaps dancing around the edges of the specs more then Acid2 may have, but I don’t think that fact renders the test as bunk. Neither test was intended as a complete test suite, only as a spot check of some web functionality and techniques.

I also sympathize with the notion that Acid2 may be more practical to present day web builders. But overshadowing that fact I think is the fact that I see the nature of the interaction of of the document with CSS vs. that of JavaScript that results in a different end composition.

Basic test suites or unit testing against individual CSS properties can leave a lot to be desired — most of the implementation problems arise in the complex interaction of the different rules and that’s where Acid2 shined [repositioning, floating, etc]. On the flip side, test suites that hit individual JavaScript functions show a much more complete picture. There what is needed isn’t to show the compounding of many common functions or actions, but instead this dance around the edges and spot checking different areas of support that may just not have everyday use that I think is seen in Acid3.

#8 On April 7th, 2008 4:25 pm David Storey replied:

I’d like to reply to a few points, and reinforce what Liorean has said. Opera and WebKit haven’t yet passed the Acid3 test, although we are both close. Both browsers have announced (on the same day) that the standards part of the test has been passed (the DOM and rendering tests) but not the performance tests. I believe the first two are the most important, as both Opera and WebKit are fast anyway.

In response to Simon, your statement is simply not true. We (Opera) were already working on standards support before Acid3 was published (there is a graph somewhere showing the progress of all browsers stating before the test even existed – and Opera’s score rises well before we started working on Acid3). For everything tested in Acid3, we support correctly (not to say bug free) and have workable real world support. Try the CSS3.info selectors test or Daniel Glazman’s selectors test, Opera’s SVG support or DOM Level 2 Events for example.

Where this point probably came from is WebKits hacking in of SMIL animation,which is currently unusable except to pass the test. This is very true, but I’d be surprised if they release a version of Safari with SMIL support at that level. They will either disable it or improve their support.

In response to Dušan, that statement you read is pure FUD. As Liorean said, the release that passes the standards part of the Acid3 test is using our internal testing system, called Gogi. We released it like that because the ACID3 work was done on a Core version after our next release, so there was no desktop build with it included. Gogi does not suppose to be used as a browser on the real web and the Opera Labs post says as much. A version of that engine will go into some undecided versions of Opera in the future. It was released to let people see that we did in fact pass the parts of the test we claim we passed. As such it is a proof of concept.

I’m just looking forward to seeing which browser will have a release version passing the standards part of the test, and which browser out of Firefox or IE will come in third. It’ll be a great day when IE can handle CSS3 selectors and DOM 2 Events, not to mention SVG, RGBA and Web Fonts.

#9 On April 8th, 2008 6:02 am exclipy replied:

“When we released Acid2, the first browser passed it in about a fortnight.”

Where’s the reference for this statement? According to Wikipedia, Acid2 was out for over 6 months before Safari became the first browser to pass it.

#10 On April 10th, 2008 2:57 am Big Brother replied:

Not sure if the latest Internet Explorer is up to the test, just know IE version 6.0 failed the link test miserably.

12/100

#11 On April 10th, 2008 5:46 am Wczasy replied:

Ian Hickson did a good job. I wonder when IE and Firefox will pass Acid 3

#12 On April 10th, 2008 1:09 pm Ed Hardy replied:

I’m really looking forward to the time the first actual release of a browser passis acid3. I guess it’ll be opera, but we’ll see.

#13 On April 10th, 2008 10:19 pm Dave replied:

Whats interesting is that Opera is leading the way on trying to get their browsers up to snuff. The others?

#14 On April 11th, 2008 3:04 am Lars Gunther replied:

Lessons learned in making tests. Every criteria must have a visual clue that says if it fails or not. The smoothness criteria have not got such a clue and people get it wrong all the time, claiming that browsers pass, even when the very people who program the browsers clearly say that they do not – yet!

#15 On April 22nd, 2008 3:05 am chiff replied:

It seems that there’s a problem with acid3 test : the page http://acid3.acidtests.org/ itself doesn’t pass the w3C CSS test
http://jigsaw.w3.org/css-validator/validator?uri=http%3A%2F%2Facid3.acidtests.org%2F&profile=css21&usermedium=all&warning=1&lang=fr

#16 On April 22nd, 2008 6:13 pm Jordan Clark replied:

@ chiff:

I’m no expert on this matter, but I think that it is this way by design. I’m absolutely sure that Ian Hickson could serve a valid style sheet if need be!

Also, I notice that the CSS validator was set to the CSS 2.1 profile in the link you provided, resulting in 14 errors; however, many of the properties are CSS 3; a re-run of the test with this profile results in only 6 errors, which are (I assume) deliberate (see results).

#17 On May 25th, 2008 3:37 pm CableCom replied:

Firefox 3.0 RC1 does not pass the test. What about the final version? Opera passes the test as a beta version.

#18 On May 25th, 2008 11:46 pm diablofan replied:

The WinGogi Desktop build from Operas Dev team DOESN’T PASS. I’ve ran the test 10 times in a row and it always failed

Proof

Yes I know I altered the tabs, but the rest is Original I swear it

#19 On June 3rd, 2008 4:46 am Ed Hardy replied:

wrong! Firefox 3.0 RC1 does pass the test! i would like to see the safari team would do it,too

#20 On June 3rd, 2008 7:53 pm Thinboy00 replied:

Many people want IE to pass. I’ve got news. IE7 still doesn’t pass Acid2, isn’t particularly or notably better than IE6 (just slightly different) in terms of coming close to passing, and its development team (or some leader or something…) has publicly stated they do not plan on worrying about standards support any time soon… so obviously if it did pass it would be due to one of the following:
1) An incredible compiler bug
2) A fake Acid3
3) Microsoft dropping everything (i.e. Vista) and focusing on standards compliance, for a change
4) An act of God

I wouldn’t hold my breath.

#21 On June 5th, 2008 11:27 am ToSensei replied:

How did the w3c-guys know the reference-rendering has no errors?
Did they calculate it in their heads, step by step, each time looking up if they “thought” it correct?
Or do they have a rendering engine especially for creating the reference rendering?
If yes, why don’t they release the source to improve every browser?

Sorry if I postet something stupid, but that’s a question not leaving my mind.

#22 On June 12th, 2008 6:10 am Pif replied:

@ToSensei :

Chicken-egg paradox here? Not quite.

I cannot specifically answer about the ACID tests, but the way every code test is done is by expecting an output according to the specs. W3C’s goal is certainly not to code a web engine, but you can use the specs as a pseudo language. That’s how you judge an implementation – what other way is there?

Wether they have made a good job of it or not is another point, but the test’s popularity and exposure should ensure pretty complete reviewing. What’s more, if there’s a flaw in the test, there’s a good chance MS will blame its failure to pass it on that!

#23 On June 16th, 2008 3:20 am Ujjwol replied:

opera and webkit(safari) didn’t pass the acid3 test in my testing but firefox got 78/100 and even Konquor doesn’t pass

#24 On June 18th, 2008 3:28 pm Maxx Kredit replied:

So many people said Acid3 will be a hard work for the most browser when it was published in March. Three and a half weeks later the hurdle is taken – by two browser.
The race around the test shows also its true meaning: to motivate the browser manufacturers for a better support in Webstandards too.

#25 On June 21st, 2008 6:44 pm Travis replied:

Latest nightly build of WebKit (the engine Safari uses) gets everything but test 26 perfect (which passes, but is below 30fps).

Is there a reason people have been challenging Safari to catch up to browsers like Firefox and IE when it’s always been far ahead in standards support?

#26 On June 25th, 2008 1:07 am Florian replied:

webkit passes the test here. Looks pretty smooth to me too using svn r32442 (libwebkit-1.0-1 version in debian testing)

with gecko 1.8 i got 53 and a completely different image. With 1.9 (ff3) 71/100.

screenshots:
http://static.ludwigf.org/stuff/acid3/gecko1.8.png
http://static.ludwigf.org/stuff/acid3/gecko1.9.png
http://static.ludwigf.org/stuff/acid3/webkit.png

Post a Reply

Comments are closed.