I have a whole page of information and pictures of Scotty's Castle that may easily been the model for Charlie Chan's Castle in the Desert.
The link to the page is http://tommenterprises.tripod.com/id113.html - or click on the website link below.
There are links to the National Parks Service entry of the Castle (courtesy of Evan Thompson), the Historic American Building Survey . . . oh, all sorts of links with all sorts of pictures and all!
Check it out and you won't ever watch Castle in the Desert the same way again!