Nailed it, Lars: "In a way, they are examples of the person in the Chinese room as described by Searle." That's too how I think about these models.

They seem to display understanding but they probably don't understand a thing about what they say or the world around them.