etree: Defining the default namespace prefix (`""`) with register_namespace has issues

[danifus]

Bug report

Bug description:

Granted the register_namespace() functionality should be used for 'well known namespace prefixes' but if the the default namespace prefix ("") is registered, the following issues can occur:

Duplicate default namespace attrs xmlns=''

import xml.etree.ElementTree as ET
ET.register_namespace("", "default")
e = ET.Element("{default}elem")
print(ET.tostring(e, default_namespace="otherdefault"))
# b'<elem xmlns="otherdefault" xmlns="default" />'

Incorrect serialisation (the noPrefixElem should raise an error but now it looks like it is in the default namespace):

import xml.etree.ElementTree as ET
ET.register_namespace("", "default")
e = ET.Element("{default}elem")
ET.SubElement(e, "noPrefixElem")
print(ET.tostring(e))
# b'<elem xmlns="default"><noPrefixElem /></elem>'

The two approaches to address this that I can think of are:

• Raise an error if the default namespace is passed to register_namespace()
  • Lucky users that register "" but don't use the default prefix's URI anywhere or has every element qualified will have working code with no issues. This would cause their working code to break.
• Handle it properly in ElementTree._namespaces() by setting the default_namespace var in that function from the global registry if the default_namespace argument is None
  • Existing code that 'works' may raise errors about ValueError: cannot use non-qualified names with default_namespace option, but at least erroneous xml would no longer be emitted
  • It may be possible to make it so code that was luckily unaffected still doesn't raise an error
  • May result in ElementTree incorrectly refuses to write attributes without namespaces when default_namespace is used #61290

I'm happy to give the second option a try (with implementing any changes required for #61290) as I've also been looking into #57587 that also needs to handle potentially multiple definitions of the default_namespace. My current thinking is that if default_namespace is provided to _namespaces(), it takes precedence even if the default namespace is also defined in the global registry.

CPython versions tested on:

3.12, CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-118416: Fix possible etree issues when registering default namespaces #118417

[joooeey]

Another issue with the default namespace prefix has been reported on StackOverflow: a namespace defined for an element somewhere inside the XML tree is moved to the root element, pulling all other elements into the same namespace.

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <runtime>
    <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
      <dependentAssembly>
        <assemblyIdentity name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" culture="neutral" />
        <bindingRedirect oldVersion="0.0.0.0-6.0.0.0" newVersion="7.0.0.0" />
      </dependentAssembly>
    </assemblyBinding>
  </runtime>
  <appSettings>
    <add key="foo" value="default"/>
    ...
  </appSettings>
</configuration>

becomes

<configuration xmlns="urn:schemas-microsoft-com:asm.v1">
 <runtime>
  <assemblyBinding>
    <dependentAssembly>
      <assemblyIdentity culture="neutral" name="Newtonsoft.Json" publicKeyToken="30ad4fe6b2a6aeed" />
      <bindingRedirect newVersion="7.0.0.0" oldVersion="0.0.0.0-6.0.0.0" />
    </dependentAssembly>
  </assemblyBinding>
 </runtime>
 <appSettings>
    <add key="foo" value="default"/>
    ...
 </appSettings>
</configuration>

This issue is still present in Python 3.13.

[danifus]

The renderer

cpython/Lib/xml/etree/ElementTree.py

Lines 854 to 855 in c9f3f5b

	def _serialize_xml(write, elem, qnames, namespaces,
	short_empty_elements, **kwargs):

only supports declaring namespaces on the root element. With that restriction, it should raise an error when it encounters both an unqualified tag and a default namespace declaration anywhere in the tree. It does do this if you specify a default namespace to the serialization functions here:

cpython/Lib/xml/etree/ElementTree.py

Lines 821 to 826 in c9f3f5b

	if default_namespace:
	# FIXME: can this be handled in XML 1.0?
	raise ValueError(
	"cannot use non-qualified names with "
	"default_namespace option"
	)

but not if the default namespace is declared in the xml. This change would make it so that it would raise an exception rather than generating the wrong xml: danifus@4d7cbd4

I've also rewritten the _serialize_xml function to support declaring namespaces at any point in the tree which is used in openpyxl and would serialize this tree correctly. The main reason for that rewrite is that it adds support for incrementally serializing a tree which can be useful for minimising memory usage.

[kljnepk]

I think this is another manifestation of #118416. It also appears closely related to the attribute-namespace loss described in #113581.

When the empty prefix is registered with register_namespace("", URI), a namespace-qualified attribute can lose its prefix during serialization. In the non-collision case that changes the attribute semantics after reparsing; in the collision case below it escalates to invalid XML because two attributes with the same serialized name are emitted.

Minimal reproducer:

import xml.etree.ElementTree as ET

URI = "http://example.com"
ET.register_namespace("", URI)

e = ET.Element(f"{{{URI}}}foo")
e.set("id", "plain")
e.set(f"{{{URI}}}id", "ns")

xml_bytes = ET.tostring(e)
print(xml_bytes.decode())
ET.fromstring(xml_bytes)

Observed locally on Python 3.13.10:

<foo xmlns="http://example.com" id="plain" id="ns" />

and the final fromstring(...) fails with:

ParseError('duplicate attribute: line 1, column 43')

Related non-collision variant:

import xml.etree.ElementTree as ET

URI = "http://example.com"
ET.register_namespace("", URI)

e = ET.Element(f"{{{URI}}}foo")
e.set(f"{{{URI}}}id", "ns")

xml_bytes = ET.tostring(e)
print(xml_bytes.decode())
print(ET.fromstring(xml_bytes).attrib)

which serializes to:

<foo xmlns="http://example.com" id="ns" />

After reparsing, the attribute is unqualified, so the namespace information is lost.

My reading is that this belongs in the same empty-prefix / _namespaces family as #118416, with the non-collision behavior closely related to #113581 and the collision case above showing a stronger invalid-XML outcome.

[serhiy-storchaka]

PR #156747 (for gh-61290) fixes the first issue as a side effect: the namespaces table now maps prefixes to URIs, so the empty prefix cannot be used twice, and the registered one gives way to the default_namespace option:

>>> ET.register_namespace("", "default")
>>> e = ET.Element("{default}elem")
>>> ET.tostring(e, default_namespace="otherdefault")
b'<ns1:elem xmlns="otherdefault" xmlns:ns1="default" />'

The second issue is not addressed: without the option, a registered empty prefix is still used as the default namespace, and an unqualified name is still written as is. Your second approach looks right to me, but it changes the meaning of register_namespace("", uri), so it deserves a separate change.