x
Main Site login Signup

WPF WebControl In A Virtualizing Container

There is a small issue with the WPF webcontrol in say a TabControl. When the entire application is themed and presented using a library of styles and templates as well as databound to a viewmodel. The TabControl reuses it's SelectedContentTemplate as it switches tabs. I have a parent container named WebPageView (constructed similarly to TabView in TabbedWPFSample) which is added to a collection. This collection is bound to in xaml. The WebPageView doesn't receive a Loaded event until it is presented the first time, as the tabs switch selected states, the Unloaded events (for the previous WebPageView) are also called. This unloaded to loaded state switching is where the control is aware it's being virtualized.

Another side-effect from this is that OnApplyTemplate, while only occurring once, only happens just prior to the first execution of the Loaded event. (Personally, it just "feels" like the source of the issue).

This presents a couple problems for me:

  1. When multiple tabs "popup" at once. Only the active tab gets to render.
  2. While other tabs remain inactive, such properties like Source and Title are not updated.
  3. Should the tab remain inactive for too long, awesomium starts to throw exceptions when it does get selected, like this one:

This IWebView instance is invalid. It has either been destroyed or it was never properly instantiated.

Even though it's not fatal, and looks like the view attempts to recover from this, it doesn't always react as expected. The title's have no text, address changed is never fired.

Notes:

  1. My application does not allow the spawn of new windows, everything is opened as a new tab.
  2. Most of the tabs will have the "Loading..." title, which is inserted by Awesomium, this means that it's been instanciated and can raise events. Just that the template hasn't been applied so it does not do any work.
  3. This issue has been present in both 1.7.1 and 1.7.2
  4. Popups are still throwing ABORTED messages, and sometimes land on about:blank, setting source when ABORTED is received still circumvents this most of the time, but it's actually less predictable now.

Is there a way to get around this? I don't really see a way to force every tab to remain selected long enough for awesomium to render atleast once without breaking what is so great about WPF.

PS. I love virtualization, I really disprove of turning it off or "breaking" it, and almost if not all controls I write are virtualization aware (if necessary, listens to it's loaded and unloaded events). It can have some performance impact (good or bad depending on the control) but the main benefit is memory.

EDIT: Through some testing I have found that the Loaded and Unloaded events all get called for the WebPageView instances as they are added, however their templates are not always applied. (Due to not being selected long enough I'm sure). Putting "this.ApplyTemplate()" inside the loaded event of WebPageView didn't help.

EDIT EDIT: This is the Output from spawning 4 tabs using http://jsfiddle.net/. (The first unloaded event is the jsfiddle tab unloading)

WebPageViewApplyTemplate
WebPageViewUnloaded
WebPageViewLoaded
TemplateApplied: False // False because it was applied during OnApplyTemplate
WebPageViewApplyTemplate
WebPageViewUnloaded
WebPageViewLoaded
TemplateApplied: False

As you can see here, only 2 of them had any events called upon them at all, and the second one that does (the final one to remain active). Sometimes doesn't even load at all (blank title bar and address). Of the 2 tabs that received no events. They receive these events when they become selected for the first time. :S

EDIT EDIT EDIT

I have done some testing with a "queue" system. Which only allows 1 tab to spawn per second. This has helped considerably, however some views still fail.

Here's the Output of events when all 4 tabs are successful (1 second delay between spawns):

The Boolean value here is "WebControl.IsLoaded"

LoadFrameFailed: False
LoadFrame: False
WebControlUnloaded // jsfiddle tab
WebControlLoaded
LoadFrameComplete: True
LoadFrameFailed: False
LoadFrame: False
WebControlUnloaded
WebControlLoaded
LoadFrameComplete: True
LoadFrameFailed: False
LoadFrame: False
WebControlUnloaded
WebControlLoaded
LoadFrameComplete: True
LoadFrameFailed: False
LoadFrame: False
LoadFrameComplete: False
WebControlUnloaded
WebControlLoaded

After many hours of relentless digging. I have figured out that the tabs that remain blank do NOT have their LoadingFrameFailed events called. As long as this event gets called, my ABORTED workaround will catch it and navigate to the frame URL causing the tab to load properly (even if it's not selected). If this event doesn't fire then it remains blank. (No title, no nothing) Until I explicitly send it to a new URL.

Here's the latest Output:

//in this print-out, 3 out of 4 tabs loaded correctly
//1 second wait between spawns
LoadFrameFailed http://google.ca/: False
Navigated //This is called inside LoadingFrameFailed
LoadFrame: False
LoadFrameComplete: False
WebControlUnloaded
WebControlLoaded
WebControlUnloaded
WebControlLoaded
LoadFrameFailed http://google.ca/: False
Navigated
LoadFrame: False
LoadFrameComplete: False
WebControlUnloaded
WebControlLoaded
LoadFrameFailed http://google.ca/: False
Navigated
LoadFrame: False
WebControlUnloaded
WebControlLoaded
LoadFrameComplete: True

// another output when only 1 tab worked (Also checks template application)
// no explicit wait on spawn, events are handled as they are fired.
LoadFrameFailed http://google.ca/: False
Navigated
WebControlTemplateApplied //view2
LoadFrame: False          //view2
WebControlUnloaded        //view1
WebControlLoaded          //view2
LoadFrameComplete: True   //view2
WebControlTemplateApplied //view3
WebControlUnloaded        //view2
WebControlLoaded          //view3
WebControlTemplateApplied //view4
WebControlUnloaded        //view3
WebControlLoaded          //view4
WebControlTemplateApplied //view5
WebControlUnloaded        //view4
WebControlLoaded          //view5

What is probably getting pretty apparent by now, is that as long as LoadingFrameFailed gets called. I can navigate the view properly. If not, then it remains empty. The bindings to the WebControl are set in the WebPageView during OnApplyTemplate... so the bindings exist at this point.

asked Sep 05 '13 at 09:05 PM SilverX gravatar image SilverX 646
more ▼
(comments are locked)
SilverX gravatar image SilverX Sep 06 '13 at 02:17 PM

I think this is officially my longest question ever. On any site :)

SilverX gravatar image SilverX Sep 06 '13 at 03:46 PM

I think that what is mostly happening is that some events start firing before OnApplyTemplate has a chance to subscribe to them (like LoadingFrameFailed). Since this is using the NativeView property. (I Think) The LoadingFrameFailed event is occurring before it's wrapped :(

Edit: I have tried moving things around so that the WebControl was a readonly property and created by WebPageView's constructor, the template then used ContentPresenter, this worked until I started closing tabs 'unbinding' was unsuccessful with an access violation. Right now the best way that doesn't throw any errors is to use WebTabControl to call ApplyTemplate on the view after it's items change. It still leaves tabs blank if they open too fast. and less often but still happens if I only allow 1 tab to spawn every 1.5 seconds! I'm smashing my head on this one.

10|1100 characters needed characters left

5 answers: sort voted first

I have been studying this since you first posted it; it's not being ignored. It's just quite a big subject. So okay, here we are:

It is important to understand exactly how the WPF WebControl is currently being instantiated and the issues that we need to deal with:

  • Starting with 1.7, the native view's pixel buffer is rendered on a surface that the users need to implement (offscreen views. We provide technology-specific surfaces), or on a platform specific window (HWND on MS Windows for windowed views).
  • In Awesomium.Windows.Controls (WPF), we decided to break the rendering and UI related code, from the rest of the logic, in the WebControl. This was because:
    • The WebControl's code files were becoming huge and very complex to maintain.
    • It would be impossible to have the same control support both offscreen and windowed (would need to be HwndHost to support a windowed native web-view).
    • We wanted to let users use a windowless WebView component in WPF (in an MVVM scenario) so we had to give them components (known as presenters in Awesomium.Windows.Controls) that take care only of rendering and UI in WPF.
  • The WPF WebControl is no longer a FrameworkElement (as it was in 1.6.x), it's a styled Control that uses 2 components (presenters) in its Style (either WebViewPresenter or WebViewHost depending on its ViewType), to render the native web-view.
  • In order to start rendering, the WebControl needs to access the assigned presenter. XAML triggers are used to pick a presenter in the control's Style, so we have to wait for OnApplyTemplate before we put our hands on the presenters.
  • If the ViewType is Window, this is even more serious since even if we internally create the native web-view, we cannot start using it before we assign a ParentWindow (this was however solved in 1.7.1. We assign a temporary native static until we get the WebViewHost presenter from the loaded style).
  • Even if there were no presenters (like in 1.6.x), properties, event handlers etc., on the WebControl, will be set by users on the control through XAML (possibly even in a template like in some of our samples). The values of some of these properties (like the ViewType and WebSession), have to be known to us before we instantiate the native web-view component that the control is going to wrap. Therefore, we cannot create the native web-view before all templates are loaded.

Taking these into consideration, in a scenario where the WebControl is created but not loaded for presentation at all or loaded later on, the WebControl will not instantiate a native web-view, until its loaded for presentation (equally, in WinForms where a windowed native web-view is wrapped by default, the control will not create the web-view until a Handle is created). Now in practice:

Things get a bit complicated to follow once you start with the EDITs, so you will allow me to focus on what's important here:

  • While the control has not yet created its underlying native web-view, it should not be attempting to access it. Code is designed to expect this so perhaps there's something there that we have missed. What would help us to identify it is:
    1. In Visual Studio, go to Debug -> Exceptions and in the Exceptions window that opens, expand "Common Language Runtime Exceptions", then expand System and check the System.InvalidOperationException (to break when it's thrown).
    2. Run your application and when you get the InvalidOperationException (This IWebView instance is invalid. It has either been destroyed or it was never properly instantiated.), open the the "Call Stack" window, copy the stack trace and send it to us.
  • You mention that popups are still getting a LoadingFrameFailed with ABORTED (-3). This is strange because we have fixed the issue that was there in 1.7.1 (and you can verify it using the samples). However I can justify it like so: Without an asynchronous ResourceInterceptor (it is in our plans), it is a really complicated procedure for us (in .NET) to distinguish requests that are sent to a child (popup) view. The ResourceInterceptor is called on the I/O thread and requests on new child views, are issued before ShowCreatedWebView. We have applied a very complex internal procedure to identify requests that target views that are the result of JavaScript window.open and in scenarios where multiple window.open calls are made from a page almost simultaneously, this procedure may fail. In general it is suggested that users, in their ShowCreatedWebView handler, add a kind of stopwatch that counts the time since a previous popup was opened and block any attempt to open multiple popups/tabs so fast (it should be considered a malicious operation anyway).

That said, for the important issue (virtualization), if you take a look in the source of the TabbedWPFSample, you will notice that we apply a trick to make sure that new tabs are loaded for presentation (so that they and any children like the WebControl can obtain any settings set in XAML and use them to create the underlying web-view):

  1. When we add a new tab, we call Application.Current.DoEvents. DoEvents is an extension of Application that we provide through Utilities in WPF, that does exactly what the equivalent method does in WinForms: It processes all pending operations (down to the lowest priority. There's an overload that accepts a specified DispatcherPriority). Provided that every new tab is brought to front at least once when added, this ensures that OnApplyTemplate will be called on the new tab when it's added.
  2. In the tab's OnApplyTemplate override, we call webBrowser.ApplyTemplate to stimulate a fast loading of the child's (WebControl) template. Once this is called on the WebControl, the WebControl itself also does the same with its children (presenters etc.).

This procedure will most of the times succeed in loading the tab's and all of its children templates almost immediately. Testing the TabbedWPFSample in the past in scenarios where many popups are created almost simultaneously, we did get ABORTED on some of these popups but we did not get a situation where a tab (and its children) does not load for presentation, so the trick applied to the sample seems to work well. Of course, DoEvents poses some performance issues (there will be a noticeable delay while tabs are being spawned), but it worths it.

(There you have the longest answer to the longest question)
answered Sep 06 '13 at 07:55 PM Perikles gravatar image Perikles C. Stephanidis ♦♦ 8.4k
more ▼
(comments are locked)
SilverX gravatar image SilverX Sep 06 '13 at 08:22 PM

It's great to know a little more about the internal presentation logic, thnank you.

A couple of notes, I do all my event subscribing inside of WebPabgeView's OnApplyTemplate, once this is finished, I too call the WebControl's ApplyTemplate. I also force a call to the WebPageView's ApplyTemplate in WebTabControl's OnItemsChanged override (if I don't do this, and try to close a tab that failed I get an exception). Another note, this is definitely part of the parent-child relationship, when I create a new view using the 'e.TargetURL' all 4 tabs load instantly and are all properly instanciated (obviously I don't actually want to do this).

I have yet to use the DoEvents you suggested, I will check that now.

PS. alerts fired from http://jsfiddle.net don't work but do from my API script ~.^

SilverX gravatar image SilverX Sep 06 '13 at 08:36 PM

Oh, and I do believe in blocking popups and/or limit them. I would just like for them to work like I think they would without protection and then block them lol.

Perikles gravatar image Perikles C. Stephanidis ♦♦ Sep 06 '13 at 08:53 PM

Yes, obviously I would love the same. But it's really really difficult without an asynchronous ResourceInterceptor. We already have a very complex logic applied there, that native Awesomium does not provide. It was important for us to distinguish:

  1. New views that are the result of link with target="_blank"
  2. New views that are the result of JavaScript window.open
  3. New views that are the result of an HTML form with target="_blank" and method="post"

As I said, the first requests to a new child view are sent before ShowCreatedWebView is fired. In our internal ResourceInterceptor we had to know exactly how the new view was created because: In 2 and 3 we have to let requests pass. But in 1 we have to block the requests because users protested that child views that they do not intend to wrap (they will set e.Cancel to true at ShowCreatedWebView) and therefore they will be destroyed by the core, are sending requests to the target URL before they are destroyed, giving false hits to the target URL. If it was not for that, we would let all requests pass.

10|1100 characters needed characters left

Regarding simultaneous or complex window.open calls that may lead to a LoadingFrameFailed with ABORTED (-3), you should know that we have applied several fixes that resolve the issue. We also applied many improvements to our internal JavaScript Interoperation Framework. All these should be expected in the next release.

Since the initial question was referring to instantiating the WPF WebControl in a WPF virtual environment, I'm setting the first answer I had provided as accepted.

answered Sep 17 '13 at 09:23 PM Perikles gravatar image Perikles C. Stephanidis ♦♦ 8.4k
more ▼
(comments are locked)
10|1100 characters needed characters left

Okay so here's the solution that (so far) has worked 100% of the time.

After applying your suggestion to use the DoEvents extension, I still experienced the ABORTED message, but ONLY for the last (4th in my test) tab. The other 3 would remain blank.

If I pass the e.TargetURL along with e.NativeViewInstance, set it as a private field "originalSource" and wait for WebPageView's Loaded event, if WebPageView's Source property (2way bound to WebControl) is null, then I manually navigate to the original source.

I have had no tabs fail to load (unless it's actually about:blank, which I believe is automatically closed by most other browsers) and observing Task Manager the parent-child relationship (if by that you mean they exist in the same process?) remains unbroken.

OH, and for the solution, I am now ignoring the ABORTED message completely and it still works :)

//////////////////////////////////////////////////

UPDATE I am currently using this, and I am getting very predictable results.. it's also successful in stopping "about:blank" tabs from opening for no reason.

private void OnShowCreatedWebView(object sender, ShowCreatedWebViewEventArgs e) {
    WebControl view = sender as WebControl;
    WebPageView newView = null;

    if (view == null || !view.IsLive)
        return;

    if (e.IsPost) //i suppose this should be enabled... should close later?
        newView = new WebPageView(e.NewViewInstance, e.TargetURL, true);

    else if (!e.TargetURL.IsBlank()) {
        e.Cancel = true;
        newView = new WebPageView(e.TargetURL, (e.IsPopup || e.IsWindowOpen));
    }
    else e.Cancel = true;


    if (newView != null)
        WebBrowser.OpenTab.Execute(newView, this);
}
answered Sep 06 '13 at 08:57 PM SilverX gravatar image SilverX 646
more ▼
(comments are locked)
10|1100 characters needed characters left

(As an answer because it will not fit as a comment)

By parent-child relationship, we actually refer to the relationship maintained in V8 between the parent DOM window and the child DOM window (well explained in the documentation of ShowCreatedWebView).

In order to maintain this relationship, child views are rendered by the same child process, they are created by native Awesomium and passed to us and navigation to the target URL (if any) is queued immediately, before we (.NET bindings) have the chance to access and wrap the view.

So parent-child view from Awesomium's point of view is one thing (refers to the web-view components only) and parent-child relationship from V8's point of view (the one we try to maintain), is another thing:

Suppose in one page you call:

var childWindow = window.open( 'somepage.html' );
childWindow.document.write( "Hello World" );

...then in ShowCreatedWebView, you do wrap the NativeViewInstance but you get the TargetURL and you set it on the wrapped child view, the parent page will never have the chance to call document.write (relationship will be broken). Consequently, after you set Source on the child web-view, JavaScript in the initially loaded page (somepage.html), will not be able to access the parent DOM window through window.opener. The page will be reloaded therefore relationship will be broken. So even though these two views are still parent-child (they render under the same process), their DOM relationship is broken.

answered Sep 06 '13 at 09:13 PM Perikles gravatar image Perikles C. Stephanidis ♦♦ 8.4k
more ▼
(comments are locked)
SilverX gravatar image SilverX Sep 06 '13 at 09:20 PM

Thanks for clarifying.

I have actually witnessed this behavior, I could write "MAHAHAHA" to the documents that "failed" but couldn't to the ones that loaded... that raises the next question. Has the minimum amount of time that an application should wait before the next view will instanciate properly been calculated? lol I mean this doesn't have to be exact but a ballpark would be good.

SilverX gravatar image SilverX Sep 06 '13 at 09:28 PM

Okay so thank you for your both of your comments, but if I do not handle the null source or the aborted message, none of the popups load. :|

SilverX gravatar image SilverX Sep 06 '13 at 09:37 PM

This behavior can be reproduced with TabbedWPFSample and http://jsfiddle.net and this javascript:

for (var i = 0; i < 4; i++)
    window.open("http://google.ca");

10|1100 characters needed characters left

If the loading issue has ben resolved, here's some more insight for you (regarding the ABORTED):

  1. A window.open command is issued in a page.
  2. In Awesomium.NET we handle this synchronously and we cache all the data passed.
  3. If a target URL is specified to the window.open call, requests are sent to the new child view.
  4. The view's identifier is specified when requests are sent. If it is not a known identifier, we assume that the request targets a new view not yet passed to us (ShowCreatedWebView is not fired yet).
  5. With the data provided with the request, we try to see if the request target's a view previously created as a result of window.open (we mainly compare target URL but other info too).
  6. If we find a match, we mark it, set the appropriate properties for user reference and we let the request pass. If not, we assume it's link with target="_blank" so we cancel the request (for the reasons I mentioned in my earlier comment).
  7. ShowCreatedWebView is called and by now we've taken care of everything. Information is provided to the user and everything internally cached is cleared.

This model generally works well but it has some flaws:

  1. The data provided to a window.open call and the data we have from a request, are not ideal for comparison. Matching may sometimes fail. For example:

    // This call will provide us with almost no data when we capture it.
    var childWindow = window.open();
    // We would normally get no calls to our ResourceInterceptor but this will send a GET 
    // request for: "http://www.amadeusoft.com/awe_wpf_5.png". This request may as well arrive
    // before ShowCreatedWebView and then there's absolutely no way to match this request
    // with the previous 'window.open' call we captured. So we will falsely assume it's a request
    // sent to a new view that is the result of a link with 'target="_blank"'.
    childWindow.document.body.innerHTML = "<img src='http://www.amadeusoft.com/awe_wpf_5.png' />";
    
  2. When multiple window.open calls are made almost simultaneously, there's always the danger we do not capture some on time (step 3 occurs before 2). We have cached no data so again we cannot match the request to a window.open call and we cancel it.

What we will provide in the next release for users to deal with this:

Right now, all this logic is executed before we call IResourceInterceptor.OnRequest and if the request is cancelled, we do not call on the interface implementation (if any) at all. What we can do is respect the users will, alongside our decision to cancel the request (we possibly add another property to ResourceRequest). So if you do not care about false hits to pages and you want requests to pass no matter what, you will be able to tell us this so no more ABORTED, and everybody is happy.

Of course things will only be almost perfect when we have an asynchronous ResourceInterceptor (that will allow us to cancel or not cancel the request, after ShowCreatedWebView is fired) and we are working on it.

EDIT (Response to comment)

  • If it sometimes aborts even single, isolated window.open calls then we fail on step 5 (in my list above). We can improve this and I add it to our issues tracker.
  • The documentation says that you should always wrap the NativeViewInstance and not set Source on the new view, when any of the IsPopup, IsWindowOpen or IsPost is true. In any other case (most possibly a link with target="_blank"), you should cancel the event (e.Cancel = true) so that the child view is destroyed, then create another view and load the TargetURL (if any). This is because in a link with target="_blank", parent-child relationship is not important and it's better for you to take advantage of the isolated processes model of Awesomium, rather than wrap the created child view and let it render under the same child process with its opener, although they have no DOM relationship.
  • What the users protested about, is that when a new view is the result of a link with target="_blank", until ShowCreatedWebView is fired and they have the chance to cancel it so that the child view is destroyed, the requests to the target URL have already been sent, giving the target URL false hits. When they later create a new view and load the target URL, this URL will actually have been loaded twice (once in the child view that we destroyed, and once in the new view they create). In some cases this is a serious issue indeed so we had to find a way to cancel requests sent to child views that are the result of a link with target="_blank".

EDIT 2

When/If we have a perfect way of identifying what's window.open or HTML form with target="_blank" and method="post" and what's a link with target="_blank" (quite difficult), we will cancel requests and destroy the child view that is the result of a link with target="_blank" internally. We will not pass the new view to ShowCreatedWebView at all. So there will be no confusions.

answered Sep 06 '13 at 10:30 PM Perikles gravatar image Perikles C. Stephanidis ♦♦ 8.4k
more ▼
(comments are locked)
SilverX gravatar image SilverX Sep 06 '13 at 10:42 PM

I'm not trying to be a bother because I understand what you're saying.

A couple of points to bring up.

  1. this happens even when using a single window.open call. Not necessarily on a loop. I only noticed it because I handled the aborted message in 1.7.1 and didn't have a problem until multiple tabs spawned at once further testing has shown that even in the updated sample that windows open in the aborted state.

  2. removed, misunderstood response

  3. are you saying I should just cancel the popup and spawn a new process or should it maintain the parent-child relationship. what's more important?

I'm confused, the documents say you should always wrap the NativeViewInstance, but if I use that then popups are always blank. it's kind of a lose lose situation.

Perikles gravatar image Perikles C. Stephanidis ♦♦ Sep 06 '13 at 11:02 PM

Answer as an EDIT.

SilverX gravatar image SilverX Sep 06 '13 at 11:20 PM

I only set the source once I have determined the load failed. I try to use the NativeViewInstance as it is used in TabbedWPFSample, I removed all my "failed" handling and the result is now 100% of my popups created with a NativeViewInstance does not load. TargetURL's (like clicking a link). Work fine.

What I'm asking is what's the point in wrapping it, if it doesn't load anyway?

SilverX gravatar image SilverX Sep 06 '13 at 11:39 PM

If I catch the aborted message and set the source, but the script that spawned the tab writes to the document the aborted message does not fire and the source is not set and keeps the relationship

Perikles gravatar image Perikles C. Stephanidis ♦♦ Sep 06 '13 at 11:55 PM +

Okay now this is getting really interesting. Just a min.

SilverX gravatar image SilverX Sep 06 '13 at 11:11 PM

Also, I have written C++/CLI bindings before infact I kind of went solo on this project (since the original was in C#):

http://spidermonkeydotnet.codeplex.com/

Would my experience be different if I moved down a level in the API and wrote my own C++/CLI wrapper or would the experience be the same?

Perikles gravatar image Perikles C. Stephanidis ♦♦ Sep 06 '13 at 11:39 PM

Awesomium is based on Chromium-WebKit-V8. Most of the wrapping/binding/extending takes place in native Awesomium. Do you want to write your own wrapper of Awesomium or of Chromium (which is actually 2 levels down)?

SilverX gravatar image SilverX Sep 06 '13 at 11:44 PM +

I mean like a .net binding to native awesomium, similar to what Awesomium.NET is but written specifically for my needs and not as a generic multi-purpose library.

I would love to obtain the license and dig through this myself but I'm a self-trained programmer and that's not within my abilities at this time.

10|1100 characters needed characters left
Your answer
toggle preview:

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.