HUD HMIS XML v.2.8 Planning

Current state of the unapproved draft HUD HMIS XML 2.8 Schema: here

Example XML instance document (mostly valid): here

Change Requests Log:

Feature Requests(Open Document Format)

Feature Requests(Microsoft Excel Format)

Discussion of Proposed Changes in HUD HMIS XML v.2.8

Extensibility During the creation of the "Minnesota version" of HUD HMIS XSD v2.7, it became apparent that the "custom" tags, while useful for their original purpose of allowing additional non-validating markup, would not address the need for the HUD HMIS Schema to be imported into a larger validating schema. Minnesota's project required exactly this when they wanted to append additional State-specific elements to the HUD HMIS XSD core. Now, there is much talk about creating a non HMIS-specific "Human Services XML", which would require this same extensibility. This should help us sidestep, for the present moment trying to implement such difficult features as releases of information. With an extensible XML, implementations can just extend HUD HMIS XML with these new features, and submit proven formats to future releases. An example of a schema extending HUD HMIS XML Schema 2.8 is here. A valid XML instance document adding the extension is here.

Preparation for Human Services XML While it is unlikely that Human Services XML will reach a state of maturity before a later revision of the HUD HMIS XSD, it may be a good idea to start preparing for a future where HUD HMIS XSD is a plug-in (import) to Human Services XML. What would this involve? Many of the requirements for being imported into Human Services XML are the same requirements for Extensibility, namely making all elements top level types so they can be imported (this is a requirement for importation in an XSD). One more difficult aspect of having HUD HMIS XML "plug" into a larger Human Services Framework is that some of the core client identifiers would no longer belong in each specific "plugged in" format, but pull up into a "Core" client identifier section, since it wouldn't make sense to have first name/last name, etc. in each subsection.

Messaging I believe it's time we propose a very simple messaging framework for HMIS XML, even if it's a draft or recommendation with the 2.8 official release. We have some examples to work from in many fields and this should be an exercise in copying. Some sort of exception report format would be helpful. The framework likely should start with a SOAP and/or REST envelope.

  1. Add/Update/Delete? Transmission Metadata in SOAP envelope
    • Implementers of past versions of HUD HMIS XML have complained that there is no way to convey whether a record should be added to the receiving database, deleted, or simply an update to an existing record. This could get complicated, as deletion would have to only reference, say, the client unique ID, so there would need to be a protocol for how to request deletion, etc... This could also be an attribute inside the XML document proper...not sure. Also, add might be redundant, since update versus delete is the only decision of true relevance. The receiving server will have to check if the record currently exists upon an add, so add/update redundantly requests the same operation. A deletion XML format could be very terse since it just has to indicate what part of what record to delete, but not necessarily specify any actual data, just indexes.
  2. New/Retry? flag in SOAP envelope. This idea came from the CARS interface.
    • Here's a  link to that. It's a great idea, I think. The idea being the sending client could state if this is the first attempt to send this or a retry. The reason I'm not sure it's useful is that the receiving web service is going to have to check if it's a resend anyway and won't be able to take the client for its word anyway. In other words, it doesn't really matter what the sending client thinks, the receiving web service still has to check for itself...
  3. Online SOAP message test facility similar to:  https://www11.hud.gov:21450/cars/CARSSOAPReceiverServlet This would allow developers and communities to independently test their end side of the web service. Of course, this won't be ready by the release of HUD HMIS XML 2.8, but is a feature that can be implemented over time. It would be very helpful to simply obtain the code for the CARSSOAPReceiverServlet.
  4. Encrypted authentication token in the SOAP envelope? Possibly also allow a request method like CARS, or just make the envelope extensible, or mention that these are the minimum elements, but more can be added on a per implementation basis.
  5. Web Services Definition Language file
    • This file tells the outside world what actions/methods are available by the web service and how to make a request for these actions. Here is HUD CARS' WSDL, a good starting point. Here is the adaptation of CARS' WSDL to HMIS: HMIS WSDL. This particular WSDL structure is a little complex, in that it allows for both having the HUD HMIS XML embedded inline in the SOAP message or attached to the SOAP envelope. It allows utilizes security tokens, and a web service operation to request that token.
  6. Exception Report - Two Different Types
    • An XML based data format will need to be defined that will convey what’s wrong with an attempted import SOAP envelope. It is critical that this is a machine and human readable file format to accommodate many types of potential users. Here are the SOAP error codes that HUD CARS has, which I think is a good starting point for us.
    • The raw HUD HMIS XML Schema validation output will be returned to the user as well. If there are no errors, the validator should convey this as well (which is the norm for XML Schema validators).

Target Namespace We need to declare a targetNamespace, so other schema can import HUD HMIS's. i.e. targetNamespace=" http://hmis.info/xsd". Along these lines, we should also create an example Schema which extends the HUD HMIS 2.8 Schema, so there is a clear methodology available or people wishing to do this. To ensure it is clear which schema are involved in any extension or even within the HUD HMIS schema, when custom tags are used, I think it would be a good idea to set elementFormDefault="qualified in the beginning "schema" tag. Then, in the instance documents, before any element tag, it must be qualified with "hmis:".

More Granularity of Timestamping The lack of timestamps in certain places is making it difficult for software implementers to match submitted records with existing records. The commonly used dateEffective and dateCollected will be added wherever it makes sense. This should make developers happy when trying to match/locate records for updating/deletion. Client Historical's overall time stamp and ID will be removed, as these will be dealt with more atomically and specifically for each data element submitted. dateEffective and dateCollected should probably be global types that can be extended to reuse their functionality throughout the schema.

Client Authorization of Information Release While this may seem at first to be a tricky addition, given the complexity of privacy rules, I think it would be worth proposing a simplified format, even if it is not ultimately included in the approved spec. It could be treated as a draft/alpha spec for comment with release 2.8, perhaps similar to how the messaging format could be treated. It would likely have to be an attribute for any element, of a type that could be extended or an extension for any type, since any client data element could be authorized for release or not.

Pull Core identifiers out into a separate section This will make it so that when importing parts of the HMIS XML Schema, logical sections can be imported, leaving the rest.

Chronically Homeless Determinants I think it's important to list out these determinants, so that the original reason for determination of homelessness is kept intact and auditable when exported between systems. I think the determinants can be optional at first, but over time, if it is proving very useful, we can require them. I'm sure software vendor will have strong opinions about this, so treading lightly on this, and looking for feedback.

Removal of some vestigial elements Including the redundant 'PP' Program Participation, 'Client Export', PersonID (within ProgramParticipation?) elements. Also removed the "Custom" tag because it has been superceded by full importability of top level types. The ExportHashing? element is no longer needed since each personal identifier element declares whether it is hashed or unhashed. Moved SourceDatabase? to become the new root element. The source database elements in the proposed SOAP format also make the "export" type unnecessary. Since the WSDL will be draft for awhile, I recommend leaving it in as an optional element for now.

Personal identifiers can be minOccurs = 0 This allows agencies to optionally send XML which has the personal identifiers stripped.

Camel-casing all types This is a convention in XML Schema and aids in readability by distinguishing elements from types. fourval -> fourVal

Removal of the default namespace from the schema, addition of hmis: as the target namespace's prefix, attributeFormDefault="qualified" and elementFormDefault="qualified This should reduce confusion about where each type/element is actually from, making users/developers more conscious/explicit about where each item is pulled from.

Wrap each service event within a need. Many HMIS systems and Continua track fulfillment of client needs, so this should be added.

Adding Address Fields, to accommodate disaster scenarios (Disaster XML fields).

Numerous Minor changes listed in the Feature Requests Spreadsheet

Nice-to-haves which likely won't make the 2.8 Release Program Bed Capacity (AHAR) Program Bed Utilization (AHAR) Folding the AIRS Agency-Site-Service model into the Schema, instead of only programs. Proegram resource data. Lat/Long?