Whatever happened to Win32::IE::Mechanize?

[Michael Roberts]

So sue me, maybe I'm just old, but I have a need to automate some browser-oriented stuff on Windows, I need to be able to pass the browser over to the user, and I just can't figure out how Selenium is supposed to work for me.

I turned to a tool I'd been intending to try out for years - Win32::IE::Mechanize - and to my utter surprise, found it is gone from CPAN!

Granted, it hadn't been updated since 2005 (really? it seems like yesterday...) but ... isn't it odd? Can it really be the case that I'm the only person in the world who intended to do something with IE one of these days? Because actually, OLE is kind of neat.

I've emailed Abe Timmerman, because I wouldn't mind adopting the module, but to no avail. Does anybody know about the considerations leading up to this decision? Because if there's some showstopping reason it just can't work any more, then I don't want to mess with it, but honestly, I'd like to maybe spruce it up a little and get it back onto CPAN.

Technically, I guess there's nothing stopping me from just uploading a new version given it's no longer there - but that seems really, um, is chutzpah-y a word? It just doesn't seem very polite.

What does the collective wisdom of the Monks dictate?

Update:

Now I see why Abe dropped the module entirely. In IE8, Microsoft did something very, very stupid, which effectively breaks OLE automation for IE, to wit: when you open up a new IE browser instance, IE assumes it will be running in Low Integrity Level, going out to the Internet. But if you retrieve a local page, that's Intranet, not Internet - and it's Medium Integrity Level. A single process can't change integrity levels. So IE invisibly opens a new tab in a new process and swaps it out.

That means the original one is defunct now. IE passes you a NewProcess event that can be trapped - but it doesn't contain a reference to a new OLE automation object or even the automation interface. It just contains, as far as I can tell, a scalar PID which - through some magickal Win32 manipulation - could possibly maybe perhaps be persuaded to yield a new OLE interface. Just one problem: I have no freaking clue at all how to do that, and apparently neither does anybody else I've found yet.

The Selenium team seems to have solved it in C++ for their IEDriver. That stuff is utterly impenetrable on first and second readings. That Microsofty interface junk is why I switched to Perl in the 90's in the first place, and I really didn't want to go back there.

So there are two things I could do. I could run all testing with a local microserver instance, which would be a quick and dirty solution. Or I could man up, figure out this Wininternals stuff, and get it into CPAN.

There's an MSDN article touching on the background for this. What to do about it is left as an exercise for the reader.

Update:

Holy Toledo. Put a MOTW (Mark of the Web, Google it) comment into a local file and IE will treat it as though it had come from the Protected Mode zone. You can re-zone any web content into a more restricted zone with the MOTW hack.

I've also found various stuff here that is tantalizingly close to what I need to do this right: this node for instance. I can't really figure out how to call CreatePerlObject from Perl - what are those parameters?!?

I've managed to figure out how to start a Medium IL Internet Explorer instance that can work with local files but not with remote ones (i.e. the same phenomenon in reverse). I think I've found the right interface to determine the zone of a given URL, but calling it looks daunting - Windows and Perl still don't really get along all that well.

This is getting pretty freaking interesting. But I don't have time for this! All I wanted to do was just automate Internet Explorer...

Update:

... Wait. MOTW fixes the problems with the test suite for IEMech.

OK, that's it. Win32::IE::Mechanize will be back on CPAN this week. I think I've earned it after losing a day and a half to this freaking mess Microsoft hath wrought. (I actually understand (kind of) where they're coming from - the security model is well considered and works for the userbase they have to make it work for - but did they really have to break Automation?)

[Anonymous Monk]

 [cpan://Win32::IE::Mechanize ] , [cpan://Win32 Mechanize ], [metacpan://Win32 Mechanize ] Win32::IE::Mechanize , Win32 Mechanize, Win32 Mechanize ... Hmm, bah! at least metacpan is supposed to show the backpan-ed stuff ... huh?

searching reveals author is ABELTJE so http://backpan.perl.org/authors/id/A/AB/ABELTJE/Win32-IE-Mechanize-0.009.tar.gz

FWIW Win32::IEAutomation is essentially IE::Mechanize

[Michael Roberts]

No, the API for Win32::IEAutomation is entirely different from Mechanize. And it hasn't been touched since 2007. I found a post from Gábor Szabó in 2006 asking about the differences between the two. It's a mystery. There is unbelievable cruft in CPAN going back to the last century, and yet people seem to use Win32::IE::Mechanize and it's dropped.

[Michael Roberts]

In fact, all the CPAN code I've seen so far for automating IE will fail for IE versions 8 or higher. I'll bet you haven't used IEAutomate since IE7 at most, right?

[MidLifeXis]

The modules, unless otherwise assigned, are copyright the owners. The way cpan is run, the owner of the module is able to remove the files from cpan (when, in theory, they will be pulled to backpan).

I've emailed Abe Timmerman, because I wouldn't mind adopting the module, but to no avail.

Please describe what you mean by 'but to no avail'. I don't see a reason why the module could not be adopted (assuming that there is no show stopper in the copyright). There is even an adoption procedure on CPAN.

[Michael Roberts]

"To no avail," in this case, means that he has not yet replied. I know there's an adoption procedure; I've used it before (Iterator::Simple, when Perl 5.12 broke it) - contacting the author is the first step.
What I'm wondering is whether the community knows anything about Abe's reasons for dropping the module beyond the obvious, which is that he no longer uses it and wasn't maintaining it. Because I can totally understand that, but if there was some reason beyond "I haven't looked at this for 8 years and it's time to clean up my life" it would be nice to know it before I really start getting my hands dirty with the code.
Of course, it's too late. I'm already looking at it and feeling certain parental urges, and I know those signs. It's like a puppy dog, just begging for attention, and it's just the size module I like - not too big or hairy, but truly useful.
My sister takes in cats; I seem to be starting to take in CPAN modules.

[Jim]

OK, that's it. Win32::IE::Mechanize will be back on CPAN this week.

OK, I'll repeat your original question, Michael:  Whatever happened to Win32::IE::Mechanize? I could really use it if it's working in IE8 and available to install. (See Using Perl and Win32::OLE to Convert XML/XSLT Opened in Internet Explorer to PDF Documents.)

Jim

[Michael Roberts]

OOooooh, Jim, you would not believe the rabbit hole that Windows COM turns out to be. I have nearly all the tests working under IE 7+, except for form submission. Every few weeks I get some time to look at it again, bang my head against the wall, and realize I still don't understand what's happening under the hood well enough to diagnose the problem. Then I do some more research, learn some more, resolve to fix it next time I get some time - and the cycle repeats.

I am making progress, and now that I've seen this post, I feel like maybe I should finish the tech article I'm working on now, and devote at least tomorrow to the next iteration of that cycle.

Although I note that you don't actually need form submission! So definitely, tomorrow I will at least package up the working copy I've got right now and make it available for download. Until I can submit forms, I obviously don't want to submit anything to CPAN yet.

[trivial0921]

Hey! Did you ever post this in its modified form for download somewhere? I also don't need form submission and would love to try it out.