diff options
| author | Maxim Yurchuk <maxim-yurchuk@ydb.tech> | 2025-05-30 21:14:52 +0000 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-05-30 21:14:52 +0000 |
| commit | c75cf6fa89ba44e2fa74a15232593b2e8423ed3f (patch) | |
| tree | a7f428153ad7b3109180af04a9c84af2b0ab2a16 /contrib/python | |
| parent | b21606bc4b50665ea3fdca703e13a4b4d7a44284 (diff) | |
| parent | 8728b9da66674488bde07a092040097e46de9366 (diff) | |
| download | ydb-c75cf6fa89ba44e2fa74a15232593b2e8423ed3f.tar.gz | |
Library import 250529-1108 (#19003)
Diffstat (limited to 'contrib/python')
63 files changed, 2476 insertions, 885 deletions
diff --git a/contrib/python/fonttools/.dist-info/METADATA b/contrib/python/fonttools/.dist-info/METADATA index 98d95f11584..51b093cabdc 100644 --- a/contrib/python/fonttools/.dist-info/METADATA +++ b/contrib/python/fonttools/.dist-info/METADATA @@ -1,6 +1,6 @@ Metadata-Version: 2.4 Name: fonttools -Version: 4.57.0 +Version: 4.58.0 Summary: Tools to manipulate font files Home-page: http://github.com/fonttools/fonttools Author: Just van Rossum @@ -14,11 +14,9 @@ Classifier: Environment :: Console Classifier: Environment :: Other Environment Classifier: Intended Audience :: Developers Classifier: Intended Audience :: End Users/Desktop -Classifier: License :: OSI Approved :: MIT License Classifier: Natural Language :: English Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python -Classifier: Programming Language :: Python :: 3.8 Classifier: Programming Language :: Python :: 3.9 Classifier: Programming Language :: Python :: 3.10 Classifier: Programming Language :: Python :: 3.11 @@ -28,9 +26,10 @@ Classifier: Programming Language :: Python :: 3 Classifier: Topic :: Text Processing :: Fonts Classifier: Topic :: Multimedia :: Graphics Classifier: Topic :: Multimedia :: Graphics :: Graphics Conversion -Requires-Python: >=3.8 +Requires-Python: >=3.9 Description-Content-Type: text/x-rst License-File: LICENSE +License-File: LICENSE.external Provides-Extra: ufo Requires-Dist: fs<3,>=2.2.0; extra == "ufo" Provides-Extra: lxml @@ -98,7 +97,7 @@ What is this? fonts to and from an XML text format, which is also called TTX. It supports TrueType, OpenType, AFM and to an extent Type 1 and some Mac-specific formats. The project has an `MIT open-source - licence <LICENSE>`__. + license <LICENSE>`__. | Among other things this means you can use it free of charge. `User documentation <https://fonttools.readthedocs.io/en/latest/>`_ and @@ -108,7 +107,7 @@ are available at `Read the Docs <https://fonttools.readthedocs.io/>`_. Installation ~~~~~~~~~~~~ -FontTools requires `Python <http://www.python.org/download/>`__ 3.8 +FontTools requires `Python <http://www.python.org/download/>`__ 3.9 or later. We try to follow the same schedule of minimum Python version support as NumPy (see `NEP 29 <https://numpy.org/neps/nep-0029-deprecation_policy.html>`__). @@ -178,9 +177,6 @@ are required to unlock the extra features named "ufo", etc. * `fs <https://pypi.org/pypi/fs>`__: (aka ``pyfilesystem2``) filesystem abstraction layer. - * `enum34 <https://pypi.org/pypi/enum34>`__: backport for the built-in ``enum`` - module (only required on Python < 3.4). - *Extra:* ``ufo`` - ``Lib/fontTools/ttLib/woff2.py`` @@ -345,7 +341,7 @@ How to make a new release automate that too. -Acknowledgements +Acknowledgments ~~~~~~~~~~~~~~~~ In alphabetical order: @@ -356,7 +352,7 @@ Vincent Connare, David Corbett, Simon Cozens, Dave Crossland, Simon Daniels, Peter Dekkers, Behdad Esfahbod, Behnam Esfahbod, Hannes Famira, Sam Fishman, Matt Fontaine, Takaaki Fuji, Rob Hagemans, Yannis Haralambous, Greg Hitchcock, Jeremie Hornus, Khaled Hosny, John Hudson, Denis Moyogo Jacquerye, Jack Jansen, -Tom Kacvinsky, Jens Kutilek, Antoine Leca, Werner Lemberg, Tal Leming, Peter +Tom Kacvinsky, Jens Kutilek, Antoine Leca, Werner Lemberg, Tal Leming, Liang Hai, Peter Lofting, Cosimo Lupo, Olli Meier, Masaya Nakamura, Dave Opstad, Laurence Penney, Roozbeh Pournader, Garret Rieger, Read Roberts, Colin Rofls, Guido van Rossum, Just van Rossum, Andreas Seidel, Georg Seifert, Chris Simpkins, Miguel Sousa, @@ -392,6 +388,44 @@ Have fun! Changelog ~~~~~~~~~ +4.58.0 (released 2025-05-10) +---------------------------- + +- Drop Python 3.8, require 3.9+ (#3819) +- [HVAR, VVAR] Prune unused regions when using a direct mapping (#3797) +- [Docs] Improvements to ufoLib documentation (#3721) +- [Docs] Improvements to varLib documentation (#3727) +- [Docs] Improvements to Pens and pen-module documentation (#3724) +- [Docs] Miscellany updates to docs (misc modules and smaller modules) (#3730) +- [subset] Close codepoints over BiDi mirror variants. (#3801) +- [feaLib] Fix serializing ChainContextPosStatement and + ChainContextSubstStatement in some rare cases (#3788) +- [designspaceLib] Clarify user expectations for getStatNames (#2892) +- [GVAR] Add support for new `GVAR` table (#3728) +- [TSI0, TSI5] Derive number of entries to decompile from data length (#2477) +- [ttLib] Fix `AttributeError` when reporting table overflow (#3808) +- [ttLib] Apply rounding more often in getCoordinates (#3798) +- [ttLib] Ignore component bounds if empty (#3799) +- [ttLib] Change the separator for duplicate glyph names from "#" to "." (#3809) +- [feaLib] Support subtable breaks in CursivePos, MarkBasePos, MarkToLigPos and + MarkToMarkPos lookups (#3800, #3807) +- [feaLib] If the same lookup has single substitutions and ligature + substitutions, upgrade single substitutions to ligature substitutions with + one input glyph (#3805) +- [feaLib] Correctly handle <NULL> in single pos lookups (#3803) +- [feaLib] Remove duplicates from class pair pos classes instead of raising an + error (#3804) +- [feaLib] Support creating extension lookups using useExtenion lookup flag + instead of silently ignoring it (#3811) +- [STAT] Add typing for the simpler STAT arguments (#3812) +- [otlLib.builder] Add future import for annotations (#3814) +- [cffLib] Fix reading supplement encoding (#3813) +- [voltLib] Add some missing functionality and fixes to voltLib and VoltToFea, + making the conversion to feature files more robust. Add also `fonttools + voltLib` command line tool to compile VOLT sources directly (doing an + intermediate fea conversion internally) (#3818) +- [pens] Add some PointPen annotations (#3820) + 4.57.0 (released 2025-04-03) ---------------------------- diff --git a/contrib/python/fonttools/README.rst b/contrib/python/fonttools/README.rst index b604ea7ca59..e40554dae8f 100644 --- a/contrib/python/fonttools/README.rst +++ b/contrib/python/fonttools/README.rst @@ -8,7 +8,7 @@ What is this? fonts to and from an XML text format, which is also called TTX. It supports TrueType, OpenType, AFM and to an extent Type 1 and some Mac-specific formats. The project has an `MIT open-source - licence <LICENSE>`__. + license <LICENSE>`__. | Among other things this means you can use it free of charge. `User documentation <https://fonttools.readthedocs.io/en/latest/>`_ and @@ -18,7 +18,7 @@ are available at `Read the Docs <https://fonttools.readthedocs.io/>`_. Installation ~~~~~~~~~~~~ -FontTools requires `Python <http://www.python.org/download/>`__ 3.8 +FontTools requires `Python <http://www.python.org/download/>`__ 3.9 or later. We try to follow the same schedule of minimum Python version support as NumPy (see `NEP 29 <https://numpy.org/neps/nep-0029-deprecation_policy.html>`__). @@ -88,9 +88,6 @@ are required to unlock the extra features named "ufo", etc. * `fs <https://pypi.org/pypi/fs>`__: (aka ``pyfilesystem2``) filesystem abstraction layer. - * `enum34 <https://pypi.org/pypi/enum34>`__: backport for the built-in ``enum`` - module (only required on Python < 3.4). - *Extra:* ``ufo`` - ``Lib/fontTools/ttLib/woff2.py`` @@ -255,7 +252,7 @@ How to make a new release automate that too. -Acknowledgements +Acknowledgments ~~~~~~~~~~~~~~~~ In alphabetical order: @@ -266,7 +263,7 @@ Vincent Connare, David Corbett, Simon Cozens, Dave Crossland, Simon Daniels, Peter Dekkers, Behdad Esfahbod, Behnam Esfahbod, Hannes Famira, Sam Fishman, Matt Fontaine, Takaaki Fuji, Rob Hagemans, Yannis Haralambous, Greg Hitchcock, Jeremie Hornus, Khaled Hosny, John Hudson, Denis Moyogo Jacquerye, Jack Jansen, -Tom Kacvinsky, Jens Kutilek, Antoine Leca, Werner Lemberg, Tal Leming, Peter +Tom Kacvinsky, Jens Kutilek, Antoine Leca, Werner Lemberg, Tal Leming, Liang Hai, Peter Lofting, Cosimo Lupo, Olli Meier, Masaya Nakamura, Dave Opstad, Laurence Penney, Roozbeh Pournader, Garret Rieger, Read Roberts, Colin Rofls, Guido van Rossum, Just van Rossum, Andreas Seidel, Georg Seifert, Chris Simpkins, Miguel Sousa, diff --git a/contrib/python/fonttools/fontTools/__init__.py b/contrib/python/fonttools/fontTools/__init__.py index bc9f0e90559..6e41eb43a8b 100644 --- a/contrib/python/fonttools/fontTools/__init__.py +++ b/contrib/python/fonttools/fontTools/__init__.py @@ -3,6 +3,6 @@ from fontTools.misc.loggingTools import configLogger log = logging.getLogger(__name__) -version = __version__ = "4.57.0" +version = __version__ = "4.58.0" __all__ = ["version", "log", "configLogger"] diff --git a/contrib/python/fonttools/fontTools/cffLib/__init__.py b/contrib/python/fonttools/fontTools/cffLib/__init__.py index d75e23b750e..4ad724a27a8 100644 --- a/contrib/python/fonttools/fontTools/cffLib/__init__.py +++ b/contrib/python/fonttools/fontTools/cffLib/__init__.py @@ -1464,10 +1464,11 @@ class CharsetConverter(SimpleConverter): if glyphName in allNames: # make up a new glyphName that's unique n = allNames[glyphName] - while (glyphName + "#" + str(n)) in allNames: + names = set(allNames) | set(charset) + while (glyphName + "." + str(n)) in names: n += 1 allNames[glyphName] = n + 1 - glyphName = glyphName + "#" + str(n) + glyphName = glyphName + "." + str(n) allNames[glyphName] = 1 newCharset.append(glyphName) charset = newCharset @@ -1663,25 +1664,26 @@ class EncodingConverter(SimpleConverter): return "StandardEncoding" elif value == 1: return "ExpertEncoding" + # custom encoding at offset `value` + assert value > 1 + file = parent.file + file.seek(value) + log.log(DEBUG, "loading Encoding at %s", value) + fmt = readCard8(file) + haveSupplement = bool(fmt & 0x80) + fmt = fmt & 0x7F + + if fmt == 0: + encoding = parseEncoding0(parent.charset, file) + elif fmt == 1: + encoding = parseEncoding1(parent.charset, file) else: - assert value > 1 - file = parent.file - file.seek(value) - log.log(DEBUG, "loading Encoding at %s", value) - fmt = readCard8(file) - haveSupplement = fmt & 0x80 - if haveSupplement: - raise NotImplementedError("Encoding supplements are not yet supported") - fmt = fmt & 0x7F - if fmt == 0: - encoding = parseEncoding0( - parent.charset, file, haveSupplement, parent.strings - ) - elif fmt == 1: - encoding = parseEncoding1( - parent.charset, file, haveSupplement, parent.strings - ) - return encoding + raise ValueError(f"Unknown Encoding format: {fmt}") + + if haveSupplement: + parseEncodingSupplement(file, encoding, parent.strings) + + return encoding def write(self, parent, value): if value == "StandardEncoding": @@ -1719,27 +1721,60 @@ class EncodingConverter(SimpleConverter): return encoding -def parseEncoding0(charset, file, haveSupplement, strings): +def readSID(file): + """Read a String ID (SID) — 2-byte unsigned integer.""" + data = file.read(2) + if len(data) != 2: + raise EOFError("Unexpected end of file while reading SID") + return struct.unpack(">H", data)[0] # big-endian uint16 + + +def parseEncodingSupplement(file, encoding, strings): + """ + Parse the CFF Encoding supplement data: + - nSups: number of supplementary mappings + - each mapping: (code, SID) pair + and apply them to the `encoding` list in place. + """ + nSups = readCard8(file) + for _ in range(nSups): + code = readCard8(file) + sid = readSID(file) + name = strings[sid] + encoding[code] = name + + +def parseEncoding0(charset, file): + """ + Format 0: simple list of codes. + After reading the base table, optionally parse the supplement. + """ nCodes = readCard8(file) encoding = [".notdef"] * 256 for glyphID in range(1, nCodes + 1): code = readCard8(file) if code != 0: encoding[code] = charset[glyphID] + return encoding -def parseEncoding1(charset, file, haveSupplement, strings): +def parseEncoding1(charset, file): + """ + Format 1: range-based encoding. + After reading the base ranges, optionally parse the supplement. + """ nRanges = readCard8(file) encoding = [".notdef"] * 256 glyphID = 1 - for i in range(nRanges): + for _ in range(nRanges): code = readCard8(file) nLeft = readCard8(file) - for glyphID in range(glyphID, glyphID + nLeft + 1): + for _ in range(nLeft + 1): encoding[code] = charset[glyphID] - code = code + 1 - glyphID = glyphID + 1 + code += 1 + glyphID += 1 + return encoding diff --git a/contrib/python/fonttools/fontTools/designspaceLib/statNames.py b/contrib/python/fonttools/fontTools/designspaceLib/statNames.py index 1474e5fcf56..4e4f73470a2 100644 --- a/contrib/python/fonttools/fontTools/designspaceLib/statNames.py +++ b/contrib/python/fonttools/fontTools/designspaceLib/statNames.py @@ -12,14 +12,13 @@ instance: from __future__ import annotations from dataclasses import dataclass -from typing import Dict, Optional, Tuple, Union +from typing import Dict, Literal, Optional, Tuple, Union import logging from fontTools.designspaceLib import ( AxisDescriptor, AxisLabelDescriptor, DesignSpaceDocument, - DesignSpaceDocumentError, DiscreteAxisDescriptor, SimpleLocationDict, SourceDescriptor, @@ -27,9 +26,13 @@ from fontTools.designspaceLib import ( LOGGER = logging.getLogger(__name__) -# TODO(Python 3.8): use Literal -# RibbiStyleName = Union[Literal["regular"], Literal["bold"], Literal["italic"], Literal["bold italic"]] -RibbiStyle = str +RibbiStyleName = Union[ + Literal["regular"], + Literal["bold"], + Literal["italic"], + Literal["bold italic"], +] + BOLD_ITALIC_TO_RIBBI_STYLE = { (False, False): "regular", (False, True): "italic", @@ -46,7 +49,7 @@ class StatNames: styleNames: Dict[str, str] postScriptFontName: Optional[str] styleMapFamilyNames: Dict[str, str] - styleMapStyleName: Optional[RibbiStyle] + styleMapStyleName: Optional[RibbiStyleName] def getStatNames( @@ -61,6 +64,10 @@ def getStatNames( localized names will be empty (family and style names), or the name will be None (PostScript name). + Note: this method does not consider info attached to the instance, like + family name. The user needs to override all names on an instance that STAT + information would compute differently than desired. + .. versionadded:: 5.0 """ familyNames: Dict[str, str] = {} @@ -201,7 +208,7 @@ def _getAxisLabelsForUserLocation( def _getRibbiStyle( self: DesignSpaceDocument, userLocation: SimpleLocationDict -) -> Tuple[RibbiStyle, SimpleLocationDict]: +) -> Tuple[RibbiStyleName, SimpleLocationDict]: """Compute the RIBBI style name of the given user location, return the location of the matching Regular in the RIBBI group. diff --git a/contrib/python/fonttools/fontTools/feaLib/ast.py b/contrib/python/fonttools/fontTools/feaLib/ast.py index 10c49058c45..8479d7300d6 100644 --- a/contrib/python/fonttools/fontTools/feaLib/ast.py +++ b/contrib/python/fonttools/fontTools/feaLib/ast.py @@ -337,6 +337,76 @@ class AnonymousBlock(Statement): return res +def _upgrade_mixed_subst_statements(statements): + # https://github.com/fonttools/fonttools/issues/612 + # A multiple substitution may have a single destination, in which case + # it will look just like a single substitution. So if there are both + # multiple and single substitutions, upgrade all the single ones to + # multiple substitutions. Similarly, a ligature substitution may have a + # single source glyph, so if there are both ligature and single + # substitutions, upgrade all the single ones to ligature substitutions. + + has_single = False + has_multiple = False + has_ligature = False + for s in statements: + if isinstance(s, SingleSubstStatement): + has_single = not any([s.prefix, s.suffix, s.forceChain]) + elif isinstance(s, MultipleSubstStatement): + has_multiple = not any([s.prefix, s.suffix, s.forceChain]) + elif isinstance(s, LigatureSubstStatement): + has_ligature = not any([s.prefix, s.suffix, s.forceChain]) + + to_multiple = False + to_ligature = False + + # If we have mixed single and multiple substitutions, + # upgrade all single substitutions to multiple substitutions. + if has_single and has_multiple and not has_ligature: + to_multiple = True + + # If we have mixed single and ligature substitutions, + # upgrade all single substitutions to ligature substitutions. + elif has_single and has_ligature and not has_multiple: + to_ligature = True + + if to_multiple or to_ligature: + ret = [] + for s in statements: + if isinstance(s, SingleSubstStatement): + glyphs = s.glyphs[0].glyphSet() + replacements = s.replacements[0].glyphSet() + if len(replacements) == 1: + replacements *= len(glyphs) + for glyph, replacement in zip(glyphs, replacements): + if to_multiple: + ret.append( + MultipleSubstStatement( + s.prefix, + glyph, + s.suffix, + [replacement], + s.forceChain, + location=s.location, + ) + ) + elif to_ligature: + ret.append( + LigatureSubstStatement( + s.prefix, + [GlyphName(glyph)], + s.suffix, + replacement, + s.forceChain, + location=s.location, + ) + ) + else: + ret.append(s) + return ret + return statements + + class Block(Statement): """A block of statements: feature, lookup, etc.""" @@ -348,7 +418,8 @@ class Block(Statement): """When handed a 'builder' object of comparable interface to :class:`fontTools.feaLib.builder`, walks the statements in this block, calling the builder callbacks.""" - for s in self.statements: + statements = _upgrade_mixed_subst_statements(self.statements) + for s in statements: s.build(builder) def asFea(self, indent=""): @@ -382,8 +453,7 @@ class FeatureBlock(Block): def build(self, builder): """Call the ``start_feature`` callback on the builder object, visit all the statements in this feature, and then call ``end_feature``.""" - # TODO(sascha): Handle use_extension. - builder.start_feature(self.location, self.name) + builder.start_feature(self.location, self.name, self.use_extension) # language exclude_dflt statements modify builder.features_ # limit them to this block with temporary builder.features_ features = builder.features_ @@ -433,8 +503,7 @@ class LookupBlock(Block): self.name, self.use_extension = name, use_extension def build(self, builder): - # TODO(sascha): Handle use_extension. - builder.start_lookup_block(self.location, self.name) + builder.start_lookup_block(self.location, self.name, self.use_extension) Block.build(self, builder) builder.end_lookup_block() @@ -753,7 +822,7 @@ class ChainContextPosStatement(Statement): if len(self.suffix): res += " " + " ".join(map(asFea, self.suffix)) else: - res += " ".join(map(asFea, self.glyph)) + res += " ".join(map(asFea, self.glyphs)) res += ";" return res @@ -811,7 +880,7 @@ class ChainContextSubstStatement(Statement): if len(self.suffix): res += " " + " ".join(map(asFea, self.suffix)) else: - res += " ".join(map(asFea, self.glyph)) + res += " ".join(map(asFea, self.glyphs)) res += ";" return res @@ -1512,7 +1581,9 @@ class SinglePosStatement(Statement): res += " ".join(map(asFea, self.prefix)) + " " res += " ".join( [ - asFea(x[0]) + "'" + ((" " + x[1].asFea()) if x[1] else "") + asFea(x[0]) + + "'" + + ((" " + x[1].asFea()) if x[1] is not None else "") for x in self.pos ] ) @@ -1520,7 +1591,10 @@ class SinglePosStatement(Statement): res += " " + " ".join(map(asFea, self.suffix)) else: res += " ".join( - [asFea(x[0]) + " " + (x[1].asFea() if x[1] else "") for x in self.pos] + [ + asFea(x[0]) + " " + (x[1].asFea() if x[1] is not None else "") + for x in self.pos + ] ) res += ";" return res @@ -2103,7 +2177,7 @@ class VariationBlock(Block): def build(self, builder): """Call the ``start_feature`` callback on the builder object, visit all the statements in this feature, and then call ``end_feature``.""" - builder.start_feature(self.location, self.name) + builder.start_feature(self.location, self.name, self.use_extension) if ( self.conditionset != "NULL" and self.conditionset not in builder.conditionsets_ diff --git a/contrib/python/fonttools/fontTools/feaLib/builder.py b/contrib/python/fonttools/fontTools/feaLib/builder.py index 8b2c7208b29..1583f06d9ec 100644 --- a/contrib/python/fonttools/fontTools/feaLib/builder.py +++ b/contrib/python/fonttools/fontTools/feaLib/builder.py @@ -126,6 +126,7 @@ class Builder(object): self.script_ = None self.lookupflag_ = 0 self.lookupflag_markFilterSet_ = None + self.use_extension_ = False self.language_systems = set() self.seen_non_DFLT_script_ = False self.named_lookups_ = {} @@ -141,6 +142,7 @@ class Builder(object): self.aalt_features_ = [] # [(location, featureName)*], for 'aalt' self.aalt_location_ = None self.aalt_alternates_ = {} + self.aalt_use_extension_ = False # for 'featureNames' self.featureNames_ = set() self.featureNames_ids_ = {} @@ -247,6 +249,7 @@ class Builder(object): result = builder_class(self.font, location) result.lookupflag = self.lookupflag_ result.markFilterSet = self.lookupflag_markFilterSet_ + result.extension = self.use_extension_ self.lookups_.append(result) return result @@ -272,6 +275,7 @@ class Builder(object): self.cur_lookup_ = builder_class(self.font, location) self.cur_lookup_.lookupflag = self.lookupflag_ self.cur_lookup_.markFilterSet = self.lookupflag_markFilterSet_ + self.cur_lookup_.extension = self.use_extension_ self.lookups_.append(self.cur_lookup_) if self.cur_lookup_name_: # We are starting a lookup rule inside a named lookup block. @@ -323,7 +327,7 @@ class Builder(object): } old_lookups = self.lookups_ self.lookups_ = [] - self.start_feature(self.aalt_location_, "aalt") + self.start_feature(self.aalt_location_, "aalt", self.aalt_use_extension_) if single: single_lookup = self.get_lookup_(location, SingleSubstBuilder) single_lookup.mapping = single @@ -1054,15 +1058,22 @@ class Builder(object): else: return frozenset({("DFLT", "dflt")}) - def start_feature(self, location, name): + def start_feature(self, location, name, use_extension=False): + if use_extension and name != "aalt": + raise FeatureLibError( + "'useExtension' keyword for feature blocks is allowed only for 'aalt' feature", + location, + ) self.language_systems = self.get_default_language_systems_() self.script_ = "DFLT" self.cur_lookup_ = None self.cur_feature_name_ = name self.lookupflag_ = 0 self.lookupflag_markFilterSet_ = None + self.use_extension_ = use_extension if name == "aalt": self.aalt_location_ = location + self.aalt_use_extension_ = use_extension def end_feature(self): assert self.cur_feature_name_ is not None @@ -1071,8 +1082,9 @@ class Builder(object): self.cur_lookup_ = None self.lookupflag_ = 0 self.lookupflag_markFilterSet_ = None + self.use_extension_ = False - def start_lookup_block(self, location, name): + def start_lookup_block(self, location, name, use_extension=False): if name in self.named_lookups_: raise FeatureLibError( 'Lookup "%s" has already been defined' % name, location @@ -1086,6 +1098,7 @@ class Builder(object): self.cur_lookup_name_ = name self.named_lookups_[name] = None self.cur_lookup_ = None + self.use_extension_ = use_extension if self.cur_feature_name_ is None: self.lookupflag_ = 0 self.lookupflag_markFilterSet_ = None @@ -1094,6 +1107,7 @@ class Builder(object): assert self.cur_lookup_name_ is not None self.cur_lookup_name_ = None self.cur_lookup_ = None + self.use_extension_ = False if self.cur_feature_name_ is None: self.lookupflag_ = 0 self.lookupflag_markFilterSet_ = None @@ -1471,7 +1485,9 @@ class Builder(object): lookup = self.get_lookup_(location, PairPosBuilder) v1 = self.makeOpenTypeValueRecord(location, value1, pairPosContext=True) v2 = self.makeOpenTypeValueRecord(location, value2, pairPosContext=True) - lookup.addClassPair(location, glyphclass1, v1, glyphclass2, v2) + cls1 = tuple(sorted(set(glyphclass1))) + cls2 = tuple(sorted(set(glyphclass2))) + lookup.addClassPair(location, cls1, v1, cls2, v2) def add_specific_pair_pos(self, location, glyph1, value1, glyph2, value2): if not glyph1 or not glyph2: diff --git a/contrib/python/fonttools/fontTools/feaLib/parser.py b/contrib/python/fonttools/fontTools/feaLib/parser.py index 5f647ca0acd..451dd624119 100644 --- a/contrib/python/fonttools/fontTools/feaLib/parser.py +++ b/contrib/python/fonttools/fontTools/feaLib/parser.py @@ -1613,7 +1613,7 @@ class Parser(object): "HorizAxis.BaseScriptList", "VertAxis.BaseScriptList", ), self.cur_token_ - scripts = [(self.parse_base_script_record_(count))] + scripts = [self.parse_base_script_record_(count)] while self.next_token_ == ",": self.expect_symbol_(",") scripts.append(self.parse_base_script_record_(count)) @@ -2062,44 +2062,6 @@ class Parser(object): ) self.expect_symbol_(";") - # A multiple substitution may have a single destination, in which case - # it will look just like a single substitution. So if there are both - # multiple and single substitutions, upgrade all the single ones to - # multiple substitutions. - - # Check if we have a mix of non-contextual singles and multiples. - has_single = False - has_multiple = False - for s in statements: - if isinstance(s, self.ast.SingleSubstStatement): - has_single = not any([s.prefix, s.suffix, s.forceChain]) - elif isinstance(s, self.ast.MultipleSubstStatement): - has_multiple = not any([s.prefix, s.suffix, s.forceChain]) - - # Upgrade all single substitutions to multiple substitutions. - if has_single and has_multiple: - statements = [] - for s in block.statements: - if isinstance(s, self.ast.SingleSubstStatement): - glyphs = s.glyphs[0].glyphSet() - replacements = s.replacements[0].glyphSet() - if len(replacements) == 1: - replacements *= len(glyphs) - for i, glyph in enumerate(glyphs): - statements.append( - self.ast.MultipleSubstStatement( - s.prefix, - glyph, - s.suffix, - [replacements[i]], - s.forceChain, - location=s.location, - ) - ) - else: - statements.append(s) - block.statements = statements - def is_cur_keyword_(self, k): if self.cur_token_type_ is Lexer.NAME: if isinstance(k, type("")): # basestring is gone in Python3 diff --git a/contrib/python/fonttools/fontTools/fontBuilder.py b/contrib/python/fonttools/fontTools/fontBuilder.py index d4af38fba48..f8da717babb 100644 --- a/contrib/python/fonttools/fontTools/fontBuilder.py +++ b/contrib/python/fonttools/fontTools/fontBuilder.py @@ -714,6 +714,12 @@ class FontBuilder(object): gvar.reserved = 0 gvar.variations = variations + def setupGVAR(self, variations): + gvar = self.font["GVAR"] = newTable("GVAR") + gvar.version = 1 + gvar.reserved = 0 + gvar.variations = variations + def calcGlyphBounds(self): """Calculate the bounding boxes of all glyphs in the `glyf` table. This is usually not called explicitly by client code. diff --git a/contrib/python/fonttools/fontTools/misc/etree.py b/contrib/python/fonttools/fontTools/misc/etree.py index d0967b5f52f..743546061c4 100644 --- a/contrib/python/fonttools/fontTools/misc/etree.py +++ b/contrib/python/fonttools/fontTools/misc/etree.py @@ -56,21 +56,7 @@ except ImportError: from xml.etree.ElementTree import * _have_lxml = False - import sys - - # dict is always ordered in python >= 3.6 and on pypy - PY36 = sys.version_info >= (3, 6) - try: - import __pypy__ - except ImportError: - __pypy__ = None - _dict_is_ordered = bool(PY36 or __pypy__) - del PY36, __pypy__ - - if _dict_is_ordered: - _Attrib = dict - else: - from collections import OrderedDict as _Attrib + _Attrib = dict if isinstance(Element, type): _Element = Element @@ -221,18 +207,9 @@ except ImportError: # characters, the surrogate blocks, FFFE, and FFFF: # Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF] # Here we reversed the pattern to match only the invalid characters. - # For the 'narrow' python builds supporting only UCS-2, which represent - # characters beyond BMP as UTF-16 surrogate pairs, we need to pass through - # the surrogate block. I haven't found a more elegant solution... - UCS2 = sys.maxunicode < 0x10FFFF - if UCS2: - _invalid_xml_string = re.compile( - "[\u0000-\u0008\u000B-\u000C\u000E-\u001F\uFFFE-\uFFFF]" - ) - else: - _invalid_xml_string = re.compile( - "[\u0000-\u0008\u000B-\u000C\u000E-\u001F\uD800-\uDFFF\uFFFE-\uFFFF]" - ) + _invalid_xml_string = re.compile( + "[\u0000-\u0008\u000B-\u000C\u000E-\u001F\uD800-\uDFFF\uFFFE-\uFFFF]" + ) def _tounicode(s): """Test if a string is valid user input and decode it to unicode string diff --git a/contrib/python/fonttools/fontTools/mtiLib/__init__.py b/contrib/python/fonttools/fontTools/mtiLib/__init__.py index dbedf275e3d..e797be375b6 100644 --- a/contrib/python/fonttools/fontTools/mtiLib/__init__.py +++ b/contrib/python/fonttools/fontTools/mtiLib/__init__.py @@ -1,5 +1,3 @@ -#!/usr/bin/python - # FontDame-to-FontTools for OpenType Layout tables # # Source language spec is available at: diff --git a/contrib/python/fonttools/fontTools/otlLib/builder.py b/contrib/python/fonttools/fontTools/otlLib/builder.py index b944ea8c261..064b2fce31c 100644 --- a/contrib/python/fonttools/fontTools/otlLib/builder.py +++ b/contrib/python/fonttools/fontTools/otlLib/builder.py @@ -1,6 +1,8 @@ +from __future__ import annotations + from collections import namedtuple, OrderedDict import itertools -import os +from typing import Dict, Union from fontTools.misc.fixedTools import fixedToFloat from fontTools.misc.roundTools import otRound from fontTools import ttLib @@ -10,15 +12,15 @@ from fontTools.ttLib.tables.otBase import ( valueRecordFormatDict, OTLOffsetOverflowError, OTTableWriter, - CountReference, ) -from fontTools.ttLib.tables import otBase +from fontTools.ttLib.ttFont import TTFont from fontTools.feaLib.ast import STATNameStatement from fontTools.otlLib.optimize.gpos import ( _compression_level_from_env, compact_lookup, ) from fontTools.otlLib.error import OpenTypeLibError +from fontTools.misc.loggingTools import deprecateFunction from functools import reduce import logging import copy @@ -73,7 +75,7 @@ LOOKUP_FLAG_IGNORE_MARKS = 0x0008 LOOKUP_FLAG_USE_MARK_FILTERING_SET = 0x0010 -def buildLookup(subtables, flags=0, markFilterSet=None): +def buildLookup(subtables, flags=0, markFilterSet=None, table=None, extension=False): """Turns a collection of rules into a lookup. A Lookup (as defined in the `OpenType Spec <https://docs.microsoft.com/en-gb/typography/opentype/spec/chapter2#lookupTbl>`__) @@ -98,6 +100,8 @@ def buildLookup(subtables, flags=0, markFilterSet=None): lookup. If a mark filtering set is provided, `LOOKUP_FLAG_USE_MARK_FILTERING_SET` will be set on the lookup's flags. + table (str): The name of the table this lookup belongs to, e.g. "GPOS" or "GSUB". + extension (bool): ``True`` if this is an extension lookup, ``False`` otherwise. Returns: An ``otTables.Lookup`` object or ``None`` if there are no subtables @@ -113,8 +117,21 @@ def buildLookup(subtables, flags=0, markFilterSet=None): ), "all subtables must have the same LookupType; got %s" % repr( [t.LookupType for t in subtables] ) + + if extension: + assert table in ("GPOS", "GSUB") + lookupType = 7 if table == "GSUB" else 9 + extSubTableClass = ot.lookupTypes[table][lookupType] + for i, st in enumerate(subtables): + subtables[i] = extSubTableClass() + subtables[i].Format = 1 + subtables[i].ExtSubTable = st + subtables[i].ExtensionLookupType = st.LookupType + else: + lookupType = subtables[0].LookupType + self = ot.Lookup() - self.LookupType = subtables[0].LookupType + self.LookupType = lookupType self.LookupFlag = flags self.SubTable = subtables self.SubTableCount = len(self.SubTable) @@ -133,7 +150,7 @@ def buildLookup(subtables, flags=0, markFilterSet=None): class LookupBuilder(object): SUBTABLE_BREAK_ = "SUBTABLE_BREAK" - def __init__(self, font, location, table, lookup_type): + def __init__(self, font, location, table, lookup_type, extension=False): self.font = font self.glyphMap = font.getReverseGlyphMap() self.location = location @@ -141,6 +158,7 @@ class LookupBuilder(object): self.lookupflag = 0 self.markFilterSet = None self.lookup_index = None # assigned when making final tables + self.extension = extension assert table in ("GPOS", "GSUB") def equals(self, other): @@ -149,6 +167,7 @@ class LookupBuilder(object): and self.table == other.table and self.lookupflag == other.lookupflag and self.markFilterSet == other.markFilterSet + and self.extension == other.extension ) def inferGlyphClasses(self): @@ -160,7 +179,13 @@ class LookupBuilder(object): return {} def buildLookup_(self, subtables): - return buildLookup(subtables, self.lookupflag, self.markFilterSet) + return buildLookup( + subtables, + self.lookupflag, + self.markFilterSet, + self.table, + self.extension, + ) def buildMarkClasses_(self, marks): """{"cedilla": ("BOTTOM", ast.Anchor), ...} --> {"BOTTOM":0, "TOP":1} @@ -949,8 +974,20 @@ class CursivePosBuilder(LookupBuilder): An ``otTables.Lookup`` object representing the cursive positioning lookup. """ - st = buildCursivePosSubtable(self.attachments, self.glyphMap) - return self.buildLookup_([st]) + attachments = [{}] + for key in self.attachments: + if key[0] == self.SUBTABLE_BREAK_: + attachments.append({}) + else: + attachments[-1][key] = self.attachments[key] + subtables = [buildCursivePosSubtable(s, self.glyphMap) for s in attachments] + return self.buildLookup_(subtables) + + def add_subtable_break(self, location): + self.attachments[(self.SUBTABLE_BREAK_, location)] = ( + self.SUBTABLE_BREAK_, + self.SUBTABLE_BREAK_, + ) class MarkBasePosBuilder(LookupBuilder): @@ -985,17 +1022,25 @@ class MarkBasePosBuilder(LookupBuilder): LookupBuilder.__init__(self, font, location, "GPOS", 4) self.marks = {} # glyphName -> (markClassName, anchor) self.bases = {} # glyphName -> {markClassName: anchor} + self.subtables_ = [] + + def get_subtables_(self): + subtables_ = self.subtables_ + if self.bases or self.marks: + subtables_.append((self.marks, self.bases)) + return subtables_ def equals(self, other): return ( LookupBuilder.equals(self, other) - and self.marks == other.marks - and self.bases == other.bases + and self.get_subtables_() == other.get_subtables_() ) def inferGlyphClasses(self): - result = {glyph: 1 for glyph in self.bases} - result.update({glyph: 3 for glyph in self.marks}) + result = {} + for marks, bases in self.get_subtables_(): + result.update({glyph: 1 for glyph in bases}) + result.update({glyph: 3 for glyph in marks}) return result def build(self): @@ -1005,26 +1050,33 @@ class MarkBasePosBuilder(LookupBuilder): An ``otTables.Lookup`` object representing the mark-to-base positioning lookup. """ - markClasses = self.buildMarkClasses_(self.marks) - marks = {} - for mark, (mc, anchor) in self.marks.items(): - if mc not in markClasses: - raise ValueError( - "Mark class %s not found for mark glyph %s" % (mc, mark) - ) - marks[mark] = (markClasses[mc], anchor) - bases = {} - for glyph, anchors in self.bases.items(): - bases[glyph] = {} - for mc, anchor in anchors.items(): + subtables = [] + for subtable in self.get_subtables_(): + markClasses = self.buildMarkClasses_(subtable[0]) + marks = {} + for mark, (mc, anchor) in subtable[0].items(): if mc not in markClasses: raise ValueError( - "Mark class %s not found for base glyph %s" % (mc, glyph) + "Mark class %s not found for mark glyph %s" % (mc, mark) ) - bases[glyph][markClasses[mc]] = anchor - subtables = buildMarkBasePos(marks, bases, self.glyphMap) + marks[mark] = (markClasses[mc], anchor) + bases = {} + for glyph, anchors in subtable[1].items(): + bases[glyph] = {} + for mc, anchor in anchors.items(): + if mc not in markClasses: + raise ValueError( + "Mark class %s not found for base glyph %s" % (mc, glyph) + ) + bases[glyph][markClasses[mc]] = anchor + subtables.append(buildMarkBasePosSubtable(marks, bases, self.glyphMap)) return self.buildLookup_(subtables) + def add_subtable_break(self, location): + self.subtables_.append((self.marks, self.bases)) + self.marks = {} + self.bases = {} + class MarkLigPosBuilder(LookupBuilder): """Builds a Mark-To-Ligature Positioning (GPOS5) lookup. @@ -1061,17 +1113,25 @@ class MarkLigPosBuilder(LookupBuilder): LookupBuilder.__init__(self, font, location, "GPOS", 5) self.marks = {} # glyphName -> (markClassName, anchor) self.ligatures = {} # glyphName -> [{markClassName: anchor}, ...] + self.subtables_ = [] + + def get_subtables_(self): + subtables_ = self.subtables_ + if self.ligatures or self.marks: + subtables_.append((self.marks, self.ligatures)) + return subtables_ def equals(self, other): return ( LookupBuilder.equals(self, other) - and self.marks == other.marks - and self.ligatures == other.ligatures + and self.get_subtables_() == other.get_subtables_() ) def inferGlyphClasses(self): - result = {glyph: 2 for glyph in self.ligatures} - result.update({glyph: 3 for glyph in self.marks}) + result = {} + for marks, ligatures in self.get_subtables_(): + result.update({glyph: 2 for glyph in ligatures}) + result.update({glyph: 3 for glyph in marks}) return result def build(self): @@ -1081,18 +1141,26 @@ class MarkLigPosBuilder(LookupBuilder): An ``otTables.Lookup`` object representing the mark-to-ligature positioning lookup. """ - markClasses = self.buildMarkClasses_(self.marks) - marks = { - mark: (markClasses[mc], anchor) for mark, (mc, anchor) in self.marks.items() - } - ligs = {} - for lig, components in self.ligatures.items(): - ligs[lig] = [] - for c in components: - ligs[lig].append({markClasses[mc]: a for mc, a in c.items()}) - subtables = buildMarkLigPos(marks, ligs, self.glyphMap) + subtables = [] + for subtable in self.get_subtables_(): + markClasses = self.buildMarkClasses_(subtable[0]) + marks = { + mark: (markClasses[mc], anchor) + for mark, (mc, anchor) in subtable[0].items() + } + ligs = {} + for lig, components in subtable[1].items(): + ligs[lig] = [] + for c in components: + ligs[lig].append({markClasses[mc]: a for mc, a in c.items()}) + subtables.append(buildMarkLigPosSubtable(marks, ligs, self.glyphMap)) return self.buildLookup_(subtables) + def add_subtable_break(self, location): + self.subtables_.append((self.marks, self.ligatures)) + self.marks = {} + self.ligatures = {} + class MarkMarkPosBuilder(LookupBuilder): """Builds a Mark-To-Mark Positioning (GPOS6) lookup. @@ -1125,17 +1193,25 @@ class MarkMarkPosBuilder(LookupBuilder): LookupBuilder.__init__(self, font, location, "GPOS", 6) self.marks = {} # glyphName -> (markClassName, anchor) self.baseMarks = {} # glyphName -> {markClassName: anchor} + self.subtables_ = [] + + def get_subtables_(self): + subtables_ = self.subtables_ + if self.baseMarks or self.marks: + subtables_.append((self.marks, self.baseMarks)) + return subtables_ def equals(self, other): return ( LookupBuilder.equals(self, other) - and self.marks == other.marks - and self.baseMarks == other.baseMarks + and self.get_subtables_() == other.get_subtables_() ) def inferGlyphClasses(self): - result = {glyph: 3 for glyph in self.baseMarks} - result.update({glyph: 3 for glyph in self.marks}) + result = {} + for marks, baseMarks in self.get_subtables_(): + result.update({glyph: 3 for glyph in baseMarks}) + result.update({glyph: 3 for glyph in marks}) return result def build(self): @@ -1145,25 +1221,34 @@ class MarkMarkPosBuilder(LookupBuilder): An ``otTables.Lookup`` object representing the mark-to-mark positioning lookup. """ - markClasses = self.buildMarkClasses_(self.marks) - markClassList = sorted(markClasses.keys(), key=markClasses.get) - marks = { - mark: (markClasses[mc], anchor) for mark, (mc, anchor) in self.marks.items() - } + subtables = [] + for subtable in self.get_subtables_(): + markClasses = self.buildMarkClasses_(subtable[0]) + markClassList = sorted(markClasses.keys(), key=markClasses.get) + marks = { + mark: (markClasses[mc], anchor) + for mark, (mc, anchor) in subtable[0].items() + } + + st = ot.MarkMarkPos() + st.Format = 1 + st.ClassCount = len(markClasses) + st.Mark1Coverage = buildCoverage(marks, self.glyphMap) + st.Mark2Coverage = buildCoverage(subtable[1], self.glyphMap) + st.Mark1Array = buildMarkArray(marks, self.glyphMap) + st.Mark2Array = ot.Mark2Array() + st.Mark2Array.Mark2Count = len(st.Mark2Coverage.glyphs) + st.Mark2Array.Mark2Record = [] + for base in st.Mark2Coverage.glyphs: + anchors = [subtable[1][base].get(mc) for mc in markClassList] + st.Mark2Array.Mark2Record.append(buildMark2Record(anchors)) + subtables.append(st) + return self.buildLookup_(subtables) - st = ot.MarkMarkPos() - st.Format = 1 - st.ClassCount = len(markClasses) - st.Mark1Coverage = buildCoverage(marks, self.glyphMap) - st.Mark2Coverage = buildCoverage(self.baseMarks, self.glyphMap) - st.Mark1Array = buildMarkArray(marks, self.glyphMap) - st.Mark2Array = ot.Mark2Array() - st.Mark2Array.Mark2Count = len(st.Mark2Coverage.glyphs) - st.Mark2Array.Mark2Record = [] - for base in st.Mark2Coverage.glyphs: - anchors = [self.baseMarks[base].get(mc) for mc in markClassList] - st.Mark2Array.Mark2Record.append(buildMark2Record(anchors)) - return self.buildLookup_([st]) + def add_subtable_break(self, location): + self.subtables_.append((self.marks, self.baseMarks)) + self.marks = {} + self.baseMarks = {} class ReverseChainSingleSubstBuilder(LookupBuilder): @@ -1484,6 +1569,8 @@ class SinglePosBuilder(LookupBuilder): otValueRection: A ``otTables.ValueRecord`` used to position the glyph. """ + if otValueRecord is None: + otValueRecord = ValueRecord() if not self.can_add(glyph, otValueRecord): otherLoc = self.locations[glyph] raise OpenTypeLibError( @@ -1900,53 +1987,15 @@ def buildMarkArray(marks, glyphMap): return self +@deprecateFunction( + "use buildMarkBasePosSubtable() instead", category=DeprecationWarning +) def buildMarkBasePos(marks, bases, glyphMap): """Build a list of MarkBasePos (GPOS4) subtables. - This routine turns a set of marks and bases into a list of mark-to-base - positioning subtables. Currently the list will contain a single subtable - containing all marks and bases, although at a later date it may return the - optimal list of subtables subsetting the marks and bases into groups which - save space. See :func:`buildMarkBasePosSubtable` below. - - Note that if you are implementing a layout compiler, you may find it more - flexible to use - :py:class:`fontTools.otlLib.lookupBuilders.MarkBasePosBuilder` instead. - - Example:: - - # a1, a2, a3, a4, a5 = buildAnchor(500, 100), ... - - marks = {"acute": (0, a1), "grave": (0, a1), "cedilla": (1, a2)} - bases = {"a": {0: a3, 1: a5}, "b": {0: a4, 1: a5}} - markbaseposes = buildMarkBasePos(marks, bases, font.getReverseGlyphMap()) - - Args: - marks (dict): A dictionary mapping anchors to glyphs; the keys being - glyph names, and the values being a tuple of mark class number and - an ``otTables.Anchor`` object representing the mark's attachment - point. (See :func:`buildMarkArray`.) - bases (dict): A dictionary mapping anchors to glyphs; the keys being - glyph names, and the values being dictionaries mapping mark class ID - to the appropriate ``otTables.Anchor`` object used for attaching marks - of that class. (See :func:`buildBaseArray`.) - glyphMap: a glyph name to ID map, typically returned from - ``font.getReverseGlyphMap()``. - - Returns: - A list of ``otTables.MarkBasePos`` objects. + .. deprecated:: 4.58.0 + Use :func:`buildMarkBasePosSubtable` instead. """ - # TODO: Consider emitting multiple subtables to save space. - # Partition the marks and bases into disjoint subsets, so that - # MarkBasePos rules would only access glyphs from a single - # subset. This would likely lead to smaller mark/base - # matrices, so we might be able to omit many of the empty - # anchor tables that we currently produce. Of course, this - # would only work if the MarkBasePos rules of real-world fonts - # allow partitioning into multiple subsets. We should find out - # whether this is the case; if so, implement the optimization. - # On the other hand, a very large number of subtables could - # slow down layout engines; so this would need profiling. return [buildMarkBasePosSubtable(marks, bases, glyphMap)] @@ -1954,7 +2003,15 @@ def buildMarkBasePosSubtable(marks, bases, glyphMap): """Build a single MarkBasePos (GPOS4) subtable. This builds a mark-to-base lookup subtable containing all of the referenced - marks and bases. See :func:`buildMarkBasePos`. + marks and bases. + + Example:: + + # a1, a2, a3, a4, a5 = buildAnchor(500, 100), ... + + marks = {"acute": (0, a1), "grave": (0, a1), "cedilla": (1, a2)} + bases = {"a": {0: a3, 1: a5}, "b": {0: a4, 1: a5}} + markbaseposes = [buildMarkBasePosSubtable(marks, bases, font.getReverseGlyphMap())] Args: marks (dict): A dictionary mapping anchors to glyphs; the keys being @@ -1981,14 +2038,21 @@ def buildMarkBasePosSubtable(marks, bases, glyphMap): return self +@deprecateFunction("use buildMarkLigPosSubtable() instead", category=DeprecationWarning) def buildMarkLigPos(marks, ligs, glyphMap): """Build a list of MarkLigPos (GPOS5) subtables. - This routine turns a set of marks and ligatures into a list of mark-to-ligature - positioning subtables. Currently the list will contain a single subtable - containing all marks and ligatures, although at a later date it may return - the optimal list of subtables subsetting the marks and ligatures into groups - which save space. See :func:`buildMarkLigPosSubtable` below. + .. deprecated:: 4.58.0 + Use :func:`buildMarkLigPosSubtable` instead. + """ + return [buildMarkLigPosSubtable(marks, ligs, glyphMap)] + + +def buildMarkLigPosSubtable(marks, ligs, glyphMap): + """Build a single MarkLigPos (GPOS5) subtable. + + This builds a mark-to-base lookup subtable containing all of the referenced + marks and bases. Note that if you are implementing a layout compiler, you may find it more flexible to use @@ -2009,7 +2073,7 @@ def buildMarkLigPos(marks, ligs, glyphMap): ], # "c_t": [{...}, {...}] } - markligposes = buildMarkLigPos(marks, ligs, + markligpose = buildMarkLigPosSubtable(marks, ligs, font.getReverseGlyphMap()) Args: @@ -2024,34 +2088,6 @@ def buildMarkLigPos(marks, ligs, glyphMap): ``font.getReverseGlyphMap()``. Returns: - A list of ``otTables.MarkLigPos`` objects. - - """ - # TODO: Consider splitting into multiple subtables to save space, - # as with MarkBasePos, this would be a trade-off that would need - # profiling. And, depending on how typical fonts are structured, - # it might not be worth doing at all. - return [buildMarkLigPosSubtable(marks, ligs, glyphMap)] - - -def buildMarkLigPosSubtable(marks, ligs, glyphMap): - """Build a single MarkLigPos (GPOS5) subtable. - - This builds a mark-to-base lookup subtable containing all of the referenced - marks and bases. See :func:`buildMarkLigPos`. - - Args: - marks (dict): A dictionary mapping anchors to glyphs; the keys being - glyph names, and the values being a tuple of mark class number and - an ``otTables.Anchor`` object representing the mark's attachment - point. (See :func:`buildMarkArray`.) - ligs (dict): A mapping of ligature names to an array of dictionaries: - for each component glyph in the ligature, an dictionary mapping - mark class IDs to anchors. (See :func:`buildLigatureArray`.) - glyphMap: a glyph name to ID map, typically returned from - ``font.getReverseGlyphMap()``. - - Returns: A ``otTables.MarkLigPos`` object. """ self = ot.MarkLigPos() @@ -2706,10 +2742,18 @@ class ClassDefBuilder(object): AXIS_VALUE_NEGATIVE_INFINITY = fixedToFloat(-0x80000000, 16) AXIS_VALUE_POSITIVE_INFINITY = fixedToFloat(0x7FFFFFFF, 16) +STATName = Union[int, str, Dict[str, str]] +"""A raw name ID, English name, or multilingual name.""" + def buildStatTable( - ttFont, axes, locations=None, elidedFallbackName=2, windowsNames=True, macNames=True -): + ttFont: TTFont, + axes, + locations=None, + elidedFallbackName: Union[STATName, STATNameStatement] = 2, + windowsNames: bool = True, + macNames: bool = True, +) -> None: """Add a 'STAT' table to 'ttFont'. 'axes' is a list of dictionaries describing axes and their @@ -2900,7 +2944,13 @@ def _buildAxisValuesFormat4(locations, axes, ttFont, windowsNames=True, macNames return axisValues -def _addName(ttFont, value, minNameID=0, windows=True, mac=True): +def _addName( + ttFont: TTFont, + value: Union[STATName, STATNameStatement], + minNameID: int = 0, + windows: bool = True, + mac: bool = True, +) -> int: nameTable = ttFont["name"] if isinstance(value, int): # Already a nameID diff --git a/contrib/python/fonttools/fontTools/otlLib/optimize/gpos.py b/contrib/python/fonttools/fontTools/otlLib/optimize/gpos.py index 61ea856d96e..3edbfeb306c 100644 --- a/contrib/python/fonttools/fontTools/otlLib/optimize/gpos.py +++ b/contrib/python/fonttools/fontTools/otlLib/optimize/gpos.py @@ -1,7 +1,8 @@ import logging import os from collections import defaultdict, namedtuple -from functools import reduce +from dataclasses import dataclass +from functools import cached_property, reduce from itertools import chain from math import log2 from typing import DefaultDict, Dict, Iterable, List, Sequence, Tuple @@ -192,79 +193,58 @@ ClusteringContext = namedtuple( ) +@dataclass class Cluster: - # TODO(Python 3.7): Turn this into a dataclass - # ctx: ClusteringContext - # indices: int - # Caches - # TODO(Python 3.8): use functools.cached_property instead of the - # manually cached properties, and remove the cache fields listed below. - # _indices: Optional[List[int]] = None - # _column_indices: Optional[List[int]] = None - # _cost: Optional[int] = None - - __slots__ = "ctx", "indices_bitmask", "_indices", "_column_indices", "_cost" - - def __init__(self, ctx: ClusteringContext, indices_bitmask: int): - self.ctx = ctx - self.indices_bitmask = indices_bitmask - self._indices = None - self._column_indices = None - self._cost = None + ctx: ClusteringContext + indices_bitmask: int - @property + @cached_property def indices(self): - if self._indices is None: - self._indices = bit_indices(self.indices_bitmask) - return self._indices + return bit_indices(self.indices_bitmask) - @property + @cached_property def column_indices(self): - if self._column_indices is None: - # Indices of columns that have a 1 in at least 1 line - # => binary OR all the lines - bitmask = reduce(int.__or__, (self.ctx.lines[i] for i in self.indices)) - self._column_indices = bit_indices(bitmask) - return self._column_indices + # Indices of columns that have a 1 in at least 1 line + # => binary OR all the lines + bitmask = reduce(int.__or__, (self.ctx.lines[i] for i in self.indices)) + return bit_indices(bitmask) @property def width(self): # Add 1 because Class2=0 cannot be used but needs to be encoded. return len(self.column_indices) + 1 - @property + @cached_property def cost(self): - if self._cost is None: - self._cost = ( - # 2 bytes to store the offset to this subtable in the Lookup table above - 2 - # Contents of the subtable - # From: https://docs.microsoft.com/en-us/typography/opentype/spec/gpos#pair-adjustment-positioning-format-2-class-pair-adjustment - # uint16 posFormat Format identifier: format = 2 - + 2 - # Offset16 coverageOffset Offset to Coverage table, from beginning of PairPos subtable. - + 2 - + self.coverage_bytes - # uint16 valueFormat1 ValueRecord definition — for the first glyph of the pair (may be zero). - + 2 - # uint16 valueFormat2 ValueRecord definition — for the second glyph of the pair (may be zero). - + 2 - # Offset16 classDef1Offset Offset to ClassDef table, from beginning of PairPos subtable — for the first glyph of the pair. - + 2 - + self.classDef1_bytes - # Offset16 classDef2Offset Offset to ClassDef table, from beginning of PairPos subtable — for the second glyph of the pair. - + 2 - + self.classDef2_bytes - # uint16 class1Count Number of classes in classDef1 table — includes Class 0. - + 2 - # uint16 class2Count Number of classes in classDef2 table — includes Class 0. - + 2 - # Class1Record class1Records[class1Count] Array of Class1 records, ordered by classes in classDef1. - + (self.ctx.valueFormat1_bytes + self.ctx.valueFormat2_bytes) - * len(self.indices) - * self.width - ) - return self._cost + return ( + # 2 bytes to store the offset to this subtable in the Lookup table above + 2 + # Contents of the subtable + # From: https://docs.microsoft.com/en-us/typography/opentype/spec/gpos#pair-adjustment-positioning-format-2-class-pair-adjustment + # uint16 posFormat Format identifier: format = 2 + + 2 + # Offset16 coverageOffset Offset to Coverage table, from beginning of PairPos subtable. + + 2 + + self.coverage_bytes + # uint16 valueFormat1 ValueRecord definition — for the first glyph of the pair (may be zero). + + 2 + # uint16 valueFormat2 ValueRecord definition — for the second glyph of the pair (may be zero). + + 2 + # Offset16 classDef1Offset Offset to ClassDef table, from beginning of PairPos subtable — for the first glyph of the pair. + + 2 + + self.classDef1_bytes + # Offset16 classDef2Offset Offset to ClassDef table, from beginning of PairPos subtable — for the second glyph of the pair. + + 2 + + self.classDef2_bytes + # uint16 class1Count Number of classes in classDef1 table — includes Class 0. + + 2 + # uint16 class2Count Number of classes in classDef2 table — includes Class 0. + + 2 + # Class1Record class1Records[class1Count] Array of Class1 records, ordered by classes in classDef1. + + (self.ctx.valueFormat1_bytes + self.ctx.valueFormat2_bytes) + * len(self.indices) + * self.width + ) @property def coverage_bytes(self): diff --git a/contrib/python/fonttools/fontTools/pens/pointPen.py b/contrib/python/fonttools/fontTools/pens/pointPen.py index 93a9201c991..843d7a28d31 100644 --- a/contrib/python/fonttools/fontTools/pens/pointPen.py +++ b/contrib/python/fonttools/fontTools/pens/pointPen.py @@ -12,12 +12,14 @@ This allows the caller to provide more data for each point. For instance, whether or not a point is smooth, and its name. """ +from __future__ import annotations + import math -from typing import Any, Optional, Tuple, Dict +from typing import Any, Dict, List, Optional, Tuple from fontTools.misc.loggingTools import LogMixin -from fontTools.pens.basePen import AbstractPen, MissingComponentError, PenError from fontTools.misc.transform import DecomposedTransform, Identity +from fontTools.pens.basePen import AbstractPen, MissingComponentError, PenError __all__ = [ "AbstractPointPen", @@ -28,6 +30,14 @@ __all__ = [ "ReverseContourPointPen", ] +# Some type aliases to make it easier below +Point = Tuple[float, float] +PointName = Optional[str] +# [(pt, smooth, name, kwargs)] +SegmentPointList = List[Tuple[Optional[Point], bool, PointName, Any]] +SegmentType = Optional[str] +SegmentList = List[Tuple[SegmentType, SegmentPointList]] + class AbstractPointPen: """Baseclass for all PointPens.""" @@ -88,7 +98,7 @@ class BasePointToSegmentPen(AbstractPointPen): care of all the edge cases. """ - def __init__(self): + def __init__(self) -> None: self.currentPath = None def beginPath(self, identifier=None, **kwargs): @@ -96,7 +106,7 @@ class BasePointToSegmentPen(AbstractPointPen): raise PenError("Path already begun.") self.currentPath = [] - def _flushContour(self, segments): + def _flushContour(self, segments: SegmentList) -> None: """Override this method. It will be called for each non-empty sub path with a list @@ -124,7 +134,7 @@ class BasePointToSegmentPen(AbstractPointPen): """ raise NotImplementedError - def endPath(self): + def endPath(self) -> None: if self.currentPath is None: raise PenError("Path not begun.") points = self.currentPath @@ -134,7 +144,7 @@ class BasePointToSegmentPen(AbstractPointPen): if len(points) == 1: # Not much more we can do than output a single move segment. pt, segmentType, smooth, name, kwargs = points[0] - segments = [("move", [(pt, smooth, name, kwargs)])] + segments: SegmentList = [("move", [(pt, smooth, name, kwargs)])] self._flushContour(segments) return segments = [] @@ -162,7 +172,7 @@ class BasePointToSegmentPen(AbstractPointPen): else: points = points[firstOnCurve + 1 :] + points[: firstOnCurve + 1] - currentSegment = [] + currentSegment: SegmentPointList = [] for pt, segmentType, smooth, name, kwargs in points: currentSegment.append((pt, smooth, name, kwargs)) if segmentType is None: @@ -189,7 +199,7 @@ class PointToSegmentPen(BasePointToSegmentPen): and kwargs. """ - def __init__(self, segmentPen, outputImpliedClosingLine=False): + def __init__(self, segmentPen, outputImpliedClosingLine: bool = False) -> None: BasePointToSegmentPen.__init__(self) self.pen = segmentPen self.outputImpliedClosingLine = outputImpliedClosingLine @@ -271,14 +281,14 @@ class SegmentToPointPen(AbstractPen): PointPen protocol. """ - def __init__(self, pointPen, guessSmooth=True): + def __init__(self, pointPen, guessSmooth=True) -> None: if guessSmooth: self.pen = GuessSmoothPointPen(pointPen) else: self.pen = pointPen - self.contour = None + self.contour: Optional[List[Tuple[Point, SegmentType]]] = None - def _flushContour(self): + def _flushContour(self) -> None: pen = self.pen pen.beginPath() for pt, segmentType in self.contour: @@ -594,7 +604,6 @@ class DecomposingPointPen(LogMixin, AbstractPointPen): # if the transformation has a negative determinant, it will # reverse the contour direction of the component a, b, c, d = transformation[:4] - det = a * d - b * c if a * d - b * c < 0: pen = ReverseContourPointPen(pen) glyph.drawPoints(pen) diff --git a/contrib/python/fonttools/fontTools/subset/__init__.py b/contrib/python/fonttools/fontTools/subset/__init__.py index 8458edc3592..056ad81babe 100644 --- a/contrib/python/fonttools/fontTools/subset/__init__.py +++ b/contrib/python/fonttools/fontTools/subset/__init__.py @@ -16,6 +16,7 @@ from fontTools.subset.cff import * from fontTools.subset.svg import * from fontTools.varLib import varStore, multiVarStore # For monkey-patching from fontTools.ttLib.tables._n_a_m_e import NameRecordVisitor +from fontTools.unicodedata import mirrored import sys import struct import array @@ -2870,6 +2871,15 @@ def prune_post_subset(self, font, options): def closure_glyphs(self, s): tables = [t for t in self.tables if t.isUnicode()] + # Closure unicodes, which for now is pulling in bidi mirrored variants + if s.options.bidi_closure: + additional_unicodes = set() + for u in s.unicodes_requested: + mirror_u = mirrored(u) + if mirror_u is not None: + additional_unicodes.add(mirror_u) + s.unicodes_requested.update(additional_unicodes) + # Close glyphs for table in tables: if table.format == 14: @@ -3191,6 +3201,7 @@ class Options(object): self.font_number = -1 self.pretty_svg = False self.lazy = True + self.bidi_closure = True self.set(**kwargs) diff --git a/contrib/python/fonttools/fontTools/ttLib/tables/G_V_A_R_.py b/contrib/python/fonttools/fontTools/ttLib/tables/G_V_A_R_.py new file mode 100644 index 00000000000..889b1f2a3bd --- /dev/null +++ b/contrib/python/fonttools/fontTools/ttLib/tables/G_V_A_R_.py @@ -0,0 +1,5 @@ +from ._g_v_a_r import table__g_v_a_r + + +class table_G_V_A_R_(table__g_v_a_r): + gid_size = 3 diff --git a/contrib/python/fonttools/fontTools/ttLib/tables/T_S_I__0.py b/contrib/python/fonttools/fontTools/ttLib/tables/T_S_I__0.py index 0d0e61a1cd3..d60e783c608 100644 --- a/contrib/python/fonttools/fontTools/ttLib/tables/T_S_I__0.py +++ b/contrib/python/fonttools/fontTools/ttLib/tables/T_S_I__0.py @@ -1,4 +1,4 @@ -""" TSI{0,1,2,3,5} are private tables used by Microsoft Visual TrueType (VTT) +"""TSI{0,1,2,3,5} are private tables used by Microsoft Visual TrueType (VTT) tool to store its hinting source data. TSI0 is the index table containing the lengths and offsets for the glyph @@ -8,9 +8,13 @@ in the TSI1 table. See also https://learn.microsoft.com/en-us/typography/tools/vtt/tsi-tables """ -from . import DefaultTable +import logging import struct +from . import DefaultTable + +log = logging.getLogger(__name__) + tsi0Format = ">HHL" @@ -25,7 +29,14 @@ class table_T_S_I__0(DefaultTable.DefaultTable): numGlyphs = ttFont["maxp"].numGlyphs indices = [] size = struct.calcsize(tsi0Format) - for i in range(numGlyphs + 5): + numEntries = len(data) // size + if numEntries != numGlyphs + 5: + diff = numEntries - numGlyphs - 5 + log.warning( + "Number of glyphPrograms differs from the number of glyphs in the font " + f"by {abs(diff)} ({numEntries - 5} programs vs. {numGlyphs} glyphs)." + ) + for _ in range(numEntries): glyphID, textLength, textOffset = fixlongs( *struct.unpack(tsi0Format, data[:size]) ) diff --git a/contrib/python/fonttools/fontTools/ttLib/tables/T_S_I__5.py b/contrib/python/fonttools/fontTools/ttLib/tables/T_S_I__5.py index 24078b22561..6afd76832fe 100644 --- a/contrib/python/fonttools/fontTools/ttLib/tables/T_S_I__5.py +++ b/contrib/python/fonttools/fontTools/ttLib/tables/T_S_I__5.py @@ -1,4 +1,4 @@ -""" TSI{0,1,2,3,5} are private tables used by Microsoft Visual TrueType (VTT) +"""TSI{0,1,2,3,5} are private tables used by Microsoft Visual TrueType (VTT) tool to store its hinting source data. TSI5 contains the VTT character groups. @@ -6,22 +6,33 @@ TSI5 contains the VTT character groups. See also https://learn.microsoft.com/en-us/typography/tools/vtt/tsi-tables """ +import array +import logging +import sys + from fontTools.misc.textTools import safeEval + from . import DefaultTable -import sys -import array + +log = logging.getLogger(__name__) class table_T_S_I__5(DefaultTable.DefaultTable): def decompile(self, data, ttFont): numGlyphs = ttFont["maxp"].numGlyphs - assert len(data) == 2 * numGlyphs a = array.array("H") a.frombytes(data) if sys.byteorder != "big": a.byteswap() self.glyphGrouping = {} - for i in range(numGlyphs): + numEntries = len(data) // 2 + if numEntries != numGlyphs: + diff = numEntries - numGlyphs + log.warning( + "Number of entries differs from the number of glyphs in the font " + f"by {abs(diff)} ({numEntries} entries vs. {numGlyphs} glyphs)." + ) + for i in range(numEntries): self.glyphGrouping[ttFont.getGlyphName(i)] = a[i] def compile(self, ttFont): diff --git a/contrib/python/fonttools/fontTools/ttLib/tables/__init__.py b/contrib/python/fonttools/fontTools/ttLib/tables/__init__.py index e622f1d1349..b111097a804 100644 --- a/contrib/python/fonttools/fontTools/ttLib/tables/__init__.py +++ b/contrib/python/fonttools/fontTools/ttLib/tables/__init__.py @@ -23,6 +23,7 @@ def _moduleFinderHint(): from . import G_P_K_G_ from . import G_P_O_S_ from . import G_S_U_B_ + from . import G_V_A_R_ from . import G__l_a_t from . import G__l_o_c from . import H_V_A_R_ diff --git a/contrib/python/fonttools/fontTools/ttLib/tables/_c_v_t.py b/contrib/python/fonttools/fontTools/ttLib/tables/_c_v_t.py index 92c50a1b8d6..51e2f78df8d 100644 --- a/contrib/python/fonttools/fontTools/ttLib/tables/_c_v_t.py +++ b/contrib/python/fonttools/fontTools/ttLib/tables/_c_v_t.py @@ -21,6 +21,8 @@ class table__c_v_t(DefaultTable.DefaultTable): self.values = values def compile(self, ttFont): + if not hasattr(self, "values"): + return b"" values = self.values[:] if sys.byteorder != "big": values.byteswap() diff --git a/contrib/python/fonttools/fontTools/ttLib/tables/_f_p_g_m.py b/contrib/python/fonttools/fontTools/ttLib/tables/_f_p_g_m.py index ba8e0488deb..c21a9d4b681 100644 --- a/contrib/python/fonttools/fontTools/ttLib/tables/_f_p_g_m.py +++ b/contrib/python/fonttools/fontTools/ttLib/tables/_f_p_g_m.py @@ -20,7 +20,9 @@ class table__f_p_g_m(DefaultTable.DefaultTable): self.program = program def compile(self, ttFont): - return self.program.getBytecode() + if hasattr(self, "program"): + return self.program.getBytecode() + return b"" def toXML(self, writer, ttFont): self.program.toXML(writer, ttFont) diff --git a/contrib/python/fonttools/fontTools/ttLib/tables/_g_l_y_f.py b/contrib/python/fonttools/fontTools/ttLib/tables/_g_l_y_f.py index c05fcea5d35..ea46c9f7971 100644 --- a/contrib/python/fonttools/fontTools/ttLib/tables/_g_l_y_f.py +++ b/contrib/python/fonttools/fontTools/ttLib/tables/_g_l_y_f.py @@ -1225,7 +1225,7 @@ class Glyph(object): if boundsDone is not None: boundsDone.add(glyphName) # empty components shouldn't update the bounds of the parent glyph - if g.numberOfContours == 0: + if g.yMin == g.yMax and g.xMin == g.xMax: continue x, y = compo.x, compo.y @@ -1285,11 +1285,7 @@ class Glyph(object): # however, if the referenced component glyph is another composite, we # must not round here but only at the end, after all the nested # transforms have been applied, or else rounding errors will compound. - if ( - round is not noRound - and g.numberOfContours > 0 - and not compo._hasOnlyIntegerTranslate() - ): + if round is not noRound and g.numberOfContours > 0: coordinates.toInt(round=round) if hasattr(compo, "firstPt"): # component uses two reference points: we apply the transform _before_ diff --git a/contrib/python/fonttools/fontTools/ttLib/tables/_g_v_a_r.py b/contrib/python/fonttools/fontTools/ttLib/tables/_g_v_a_r.py index e942beaf58b..07d3befb7aa 100644 --- a/contrib/python/fonttools/fontTools/ttLib/tables/_g_v_a_r.py +++ b/contrib/python/fonttools/fontTools/ttLib/tables/_g_v_a_r.py @@ -24,19 +24,24 @@ log = logging.getLogger(__name__) # FreeType2 source code for parsing 'gvar': # http://git.savannah.gnu.org/cgit/freetype/freetype2.git/tree/src/truetype/ttgxvar.c -GVAR_HEADER_FORMAT = """ +GVAR_HEADER_FORMAT_HEAD = """ > # big endian version: H reserved: H axisCount: H sharedTupleCount: H offsetToSharedTuples: I - glyphCount: H +""" +# In between the HEAD and TAIL lies the glyphCount, which is +# of different size: 2 bytes for gvar, and 3 bytes for GVAR. +GVAR_HEADER_FORMAT_TAIL = """ + > # big endian flags: H offsetToGlyphVariationData: I """ -GVAR_HEADER_SIZE = sstruct.calcsize(GVAR_HEADER_FORMAT) +GVAR_HEADER_SIZE_HEAD = sstruct.calcsize(GVAR_HEADER_FORMAT_HEAD) +GVAR_HEADER_SIZE_TAIL = sstruct.calcsize(GVAR_HEADER_FORMAT_TAIL) class table__g_v_a_r(DefaultTable.DefaultTable): @@ -51,6 +56,7 @@ class table__g_v_a_r(DefaultTable.DefaultTable): """ dependencies = ["fvar", "glyf"] + gid_size = 2 def __init__(self, tag=None): DefaultTable.DefaultTable.__init__(self, tag) @@ -74,20 +80,25 @@ class table__g_v_a_r(DefaultTable.DefaultTable): offsets.append(offset) compiledOffsets, tableFormat = self.compileOffsets_(offsets) + GVAR_HEADER_SIZE = GVAR_HEADER_SIZE_HEAD + self.gid_size + GVAR_HEADER_SIZE_TAIL header = {} header["version"] = self.version header["reserved"] = self.reserved header["axisCount"] = len(axisTags) header["sharedTupleCount"] = len(sharedTuples) header["offsetToSharedTuples"] = GVAR_HEADER_SIZE + len(compiledOffsets) - header["glyphCount"] = len(compiledGlyphs) header["flags"] = tableFormat header["offsetToGlyphVariationData"] = ( header["offsetToSharedTuples"] + sharedTupleSize ) - compiledHeader = sstruct.pack(GVAR_HEADER_FORMAT, header) - result = [compiledHeader, compiledOffsets] + result = [ + sstruct.pack(GVAR_HEADER_FORMAT_HEAD, header), + len(compiledGlyphs).to_bytes(self.gid_size, "big"), + sstruct.pack(GVAR_HEADER_FORMAT_TAIL, header), + ] + + result.append(compiledOffsets) result.extend(sharedTuples) result.extend(compiledGlyphs) return b"".join(result) @@ -104,6 +115,7 @@ class table__g_v_a_r(DefaultTable.DefaultTable): pointCountUnused = 0 # pointCount is actually unused by compileGlyph result.append( compileGlyph_( + self.gid_size, variations, pointCountUnused, axisTags, @@ -116,7 +128,19 @@ class table__g_v_a_r(DefaultTable.DefaultTable): def decompile(self, data, ttFont): axisTags = [axis.axisTag for axis in ttFont["fvar"].axes] glyphs = ttFont.getGlyphOrder() - sstruct.unpack(GVAR_HEADER_FORMAT, data[0:GVAR_HEADER_SIZE], self) + + # Parse the header + GVAR_HEADER_SIZE = GVAR_HEADER_SIZE_HEAD + self.gid_size + GVAR_HEADER_SIZE_TAIL + sstruct.unpack(GVAR_HEADER_FORMAT_HEAD, data[:GVAR_HEADER_SIZE_HEAD], self) + self.glyphCount = int.from_bytes( + data[GVAR_HEADER_SIZE_HEAD : GVAR_HEADER_SIZE_HEAD + self.gid_size], "big" + ) + sstruct.unpack( + GVAR_HEADER_FORMAT_TAIL, + data[GVAR_HEADER_SIZE_HEAD + self.gid_size : GVAR_HEADER_SIZE], + self, + ) + assert len(glyphs) == self.glyphCount assert len(axisTags) == self.axisCount sharedCoords = tv.decompileSharedTuples( @@ -146,7 +170,7 @@ class table__g_v_a_r(DefaultTable.DefaultTable): glyph = glyf[glyphName] numPointsInGlyph = self.getNumPoints_(glyph) return decompileGlyph_( - numPointsInGlyph, sharedCoords, axisTags, gvarData + self.gid_size, numPointsInGlyph, sharedCoords, axisTags, gvarData ) return read_item @@ -264,23 +288,42 @@ class table__g_v_a_r(DefaultTable.DefaultTable): def compileGlyph_( - variations, pointCount, axisTags, sharedCoordIndices, *, optimizeSize=True + dataOffsetSize, + variations, + pointCount, + axisTags, + sharedCoordIndices, + *, + optimizeSize=True, ): + assert dataOffsetSize in (2, 3) tupleVariationCount, tuples, data = tv.compileTupleVariationStore( variations, pointCount, axisTags, sharedCoordIndices, optimizeSize=optimizeSize ) if tupleVariationCount == 0: return b"" - result = [struct.pack(">HH", tupleVariationCount, 4 + len(tuples)), tuples, data] - if (len(tuples) + len(data)) % 2 != 0: + + offsetToData = 2 + dataOffsetSize + len(tuples) + + result = [ + tupleVariationCount.to_bytes(2, "big"), + offsetToData.to_bytes(dataOffsetSize, "big"), + tuples, + data, + ] + if (offsetToData + len(data)) % 2 != 0: result.append(b"\0") # padding return b"".join(result) -def decompileGlyph_(pointCount, sharedTuples, axisTags, data): - if len(data) < 4: +def decompileGlyph_(dataOffsetSize, pointCount, sharedTuples, axisTags, data): + assert dataOffsetSize in (2, 3) + if len(data) < 2 + dataOffsetSize: return [] - tupleVariationCount, offsetToData = struct.unpack(">HH", data[:4]) + + tupleVariationCount = int.from_bytes(data[:2], "big") + offsetToData = int.from_bytes(data[2 : 2 + dataOffsetSize], "big") + dataPos = offsetToData return tv.decompileTupleVariationStore( "gvar", @@ -289,6 +332,6 @@ def decompileGlyph_(pointCount, sharedTuples, axisTags, data): pointCount, sharedTuples, data, - 4, + 2 + dataOffsetSize, offsetToData, ) diff --git a/contrib/python/fonttools/fontTools/ttLib/tables/_p_o_s_t.py b/contrib/python/fonttools/fontTools/ttLib/tables/_p_o_s_t.py index fca0812f984..c449e5f0c03 100644 --- a/contrib/python/fonttools/fontTools/ttLib/tables/_p_o_s_t.py +++ b/contrib/python/fonttools/fontTools/ttLib/tables/_p_o_s_t.py @@ -122,13 +122,16 @@ class table__p_o_s_t(DefaultTable.DefaultTable): glyphName = psName = self.glyphOrder[i] if glyphName == "": glyphName = "glyph%.5d" % i + if glyphName in allNames: # make up a new glyphName that's unique n = allNames[glyphName] - while (glyphName + "#" + str(n)) in allNames: + # check if the exists in any of the seen names or later ones + names = set(allNames.keys()) | set(self.glyphOrder) + while (glyphName + "." + str(n)) in names: n += 1 allNames[glyphName] = n + 1 - glyphName = glyphName + "#" + str(n) + glyphName = glyphName + "." + str(n) self.glyphOrder[i] = glyphName allNames[glyphName] = 1 diff --git a/contrib/python/fonttools/fontTools/ttLib/tables/otBase.py b/contrib/python/fonttools/fontTools/ttLib/tables/otBase.py index 8df7c236b1c..582b02024b3 100644 --- a/contrib/python/fonttools/fontTools/ttLib/tables/otBase.py +++ b/contrib/python/fonttools/fontTools/ttLib/tables/otBase.py @@ -398,6 +398,7 @@ class OTTableWriter(object): self.localState = localState self.tableTag = tableTag self.parent = None + self.name = "<none>" def __setitem__(self, name, value): state = self.localState.copy() if self.localState else dict() diff --git a/contrib/python/fonttools/fontTools/ufoLib/__init__.py b/contrib/python/fonttools/fontTools/ufoLib/__init__.py index 42c06734ea9..f76938a8f15 100644 --- a/contrib/python/fonttools/fontTools/ufoLib/__init__.py +++ b/contrib/python/fonttools/fontTools/ufoLib/__init__.py @@ -204,7 +204,7 @@ class UFOReader(_UFOBaseIO): """Read the various components of a .ufo. Attributes: - path: An `os.PathLike` object pointing to the .ufo. + path: An :class:`os.PathLike` object pointing to the .ufo. validate: A boolean indicating if the data read should be validated. Defaults to `True`. @@ -891,7 +891,7 @@ class UFOWriter(UFOReader): """Write the various components of a .ufo. Attributes: - path: An `os.PathLike` object pointing to the .ufo. + path: An :class:`os.PathLike` object pointing to the .ufo. formatVersion: the UFO format version as a tuple of integers (major, minor), or as a single integer for the major digit only (minor is implied to be 0). By default, the latest formatVersion will be used; currently it is 3.0, diff --git a/contrib/python/fonttools/fontTools/ufoLib/converters.py b/contrib/python/fonttools/fontTools/ufoLib/converters.py index 88a26c616a8..4ee6b05e338 100644 --- a/contrib/python/fonttools/fontTools/ufoLib/converters.py +++ b/contrib/python/fonttools/fontTools/ufoLib/converters.py @@ -1,11 +1,33 @@ """ -Conversion functions. +Functions for converting UFO1 or UFO2 files into UFO3 format. + +Currently provides functionality for converting kerning rules +and kerning groups. Conversion is only supported _from_ UFO1 +or UFO2, and _to_ UFO3. """ # adapted from the UFO spec def convertUFO1OrUFO2KerningToUFO3Kerning(kerning, groups, glyphSet=()): + """Convert kerning data in UFO1 or UFO2 syntax into UFO3 syntax. + + Args: + kerning: + A dictionary containing the kerning rules defined in + the UFO font, as used in :class:`.UFOReader` objects. + groups: + A dictionary containing the groups defined in the UFO + font, as used in :class:`.UFOReader` objects. + glyphSet: + Optional; a set of glyph objects to skip (default: None). + + Returns: + 1. A dictionary representing the converted kerning data. + 2. A copy of the groups dictionary, with all groups renamed to UFO3 syntax. + 3. A dictionary containing the mapping of old group names to new group names. + + """ # gather known kerning groups based on the prefixes firstReferencedGroups, secondReferencedGroups = findKnownKerningGroups(groups) # Make lists of groups referenced in kerning pairs. @@ -63,35 +85,54 @@ def convertUFO1OrUFO2KerningToUFO3Kerning(kerning, groups, glyphSet=()): def findKnownKerningGroups(groups): - """ - This will find kerning groups with known prefixes. - In some cases not all kerning groups will be referenced - by the kerning pairs. The algorithm for locating groups - in convertUFO1OrUFO2KerningToUFO3Kerning will miss these - unreferenced groups. By scanning for known prefixes + """Find all kerning groups in a UFO1 or UFO2 font that use known prefixes. + + In some cases, not all kerning groups will be referenced + by the kerning pairs in a UFO. The algorithm for locating + groups in :func:`convertUFO1OrUFO2KerningToUFO3Kerning` will + miss these unreferenced groups. By scanning for known prefixes, this function will catch all of the prefixed groups. - These are the prefixes and sides that are handled: + The prefixes and sides by this function are: + @MMK_L_ - side 1 @MMK_R_ - side 2 - >>> testGroups = { - ... "@MMK_L_1" : None, - ... "@MMK_L_2" : None, - ... "@MMK_L_3" : None, - ... "@MMK_R_1" : None, - ... "@MMK_R_2" : None, - ... "@MMK_R_3" : None, - ... "@MMK_l_1" : None, - ... "@MMK_r_1" : None, - ... "@MMK_X_1" : None, - ... "foo" : None, - ... } - >>> first, second = findKnownKerningGroups(testGroups) - >>> sorted(first) == ['@MMK_L_1', '@MMK_L_2', '@MMK_L_3'] - True - >>> sorted(second) == ['@MMK_R_1', '@MMK_R_2', '@MMK_R_3'] - True + as defined in the UFO1 specification. + + Args: + groups: + A dictionary containing the groups defined in the UFO + font, as read by :class:`.UFOReader`. + + Returns: + Two sets; the first containing the names of all + first-side kerning groups identified in the ``groups`` + dictionary, and the second containing the names of all + second-side kerning groups identified. + + "First-side" and "second-side" are with respect to the + writing direction of the script. + + Example:: + + >>> testGroups = { + ... "@MMK_L_1" : None, + ... "@MMK_L_2" : None, + ... "@MMK_L_3" : None, + ... "@MMK_R_1" : None, + ... "@MMK_R_2" : None, + ... "@MMK_R_3" : None, + ... "@MMK_l_1" : None, + ... "@MMK_r_1" : None, + ... "@MMK_X_1" : None, + ... "foo" : None, + ... } + >>> first, second = findKnownKerningGroups(testGroups) + >>> sorted(first) == ['@MMK_L_1', '@MMK_L_2', '@MMK_L_3'] + True + >>> sorted(second) == ['@MMK_R_1', '@MMK_R_2', '@MMK_R_3'] + True """ knownFirstGroupPrefixes = ["@MMK_L_"] knownSecondGroupPrefixes = ["@MMK_R_"] @@ -110,6 +151,27 @@ def findKnownKerningGroups(groups): def makeUniqueGroupName(name, groupNames, counter=0): + """Make a kerning group name that will be unique within the set of group names. + + If the requested kerning group name already exists within the set, this + will return a new name by adding an incremented counter to the end + of the requested name. + + Args: + name: + The requested kerning group name. + groupNames: + A list of the existing kerning group names. + counter: + Optional; a counter of group names already seen (default: 0). If + :attr:`.counter` is not provided, the function will recurse, + incrementing the value of :attr:`.counter` until it finds the + first unused ``name+counter`` combination, and return that result. + + Returns: + A unique kerning group name composed of the requested name suffixed + by the smallest available integer counter. + """ # Add a number to the name if the counter is higher than zero. newName = name if counter > 0: @@ -123,6 +185,8 @@ def makeUniqueGroupName(name, groupNames, counter=0): def test(): """ + Tests for :func:`.convertUFO1OrUFO2KerningToUFO3Kerning`. + No known prefixes. >>> testKerning = { diff --git a/contrib/python/fonttools/fontTools/ufoLib/errors.py b/contrib/python/fonttools/fontTools/ufoLib/errors.py index e05dd438b43..6cc9fec3994 100644 --- a/contrib/python/fonttools/fontTools/ufoLib/errors.py +++ b/contrib/python/fonttools/fontTools/ufoLib/errors.py @@ -10,6 +10,14 @@ class UnsupportedUFOFormat(UFOLibError): class GlifLibError(UFOLibError): + """An error raised by glifLib. + + This class is a loose backport of PEP 678, adding a :attr:`.note` + attribute that can hold additional context for errors encountered. + + It will be maintained until only Python 3.11-and-later are supported. + """ + def _add_note(self, note: str) -> None: # Loose backport of PEP 678 until we only support Python 3.11+, used for # adding additional context to errors. diff --git a/contrib/python/fonttools/fontTools/ufoLib/etree.py b/contrib/python/fonttools/fontTools/ufoLib/etree.py index 77e3c16e2b4..07b924ac852 100644 --- a/contrib/python/fonttools/fontTools/ufoLib/etree.py +++ b/contrib/python/fonttools/fontTools/ufoLib/etree.py @@ -1,5 +1,5 @@ """DEPRECATED - This module is kept here only as a backward compatibility shim -for the old ufoLib.etree module, which was moved to fontTools.misc.etree. +for the old ufoLib.etree module, which was moved to :mod:`fontTools.misc.etree`. Please use the latter instead. """ diff --git a/contrib/python/fonttools/fontTools/ufoLib/filenames.py b/contrib/python/fonttools/fontTools/ufoLib/filenames.py index 7f1af58ee84..83442f1c8ce 100644 --- a/contrib/python/fonttools/fontTools/ufoLib/filenames.py +++ b/contrib/python/fonttools/fontTools/ufoLib/filenames.py @@ -1,6 +1,22 @@ """ -User name to file name conversion. -This was taken from the UFO 3 spec. +Convert user-provided internal UFO names to spec-compliant filenames. + +This module implements the algorithm for converting between a "user name" - +something that a user can choose arbitrarily inside a font editor - and a file +name suitable for use in a wide range of operating systems and filesystems. + +The `UFO 3 specification <http://unifiedfontobject.org/versions/ufo3/conventions/>`_ +provides an example of an algorithm for such conversion, which avoids illegal +characters, reserved file names, ambiguity between upper- and lower-case +characters, and clashes with existing files. + +This code was originally copied from +`ufoLib <https://github.com/unified-font-object/ufoLib/blob/8747da7/Lib/ufoLib/filenames.py>`_ +by Tal Leming and is copyright (c) 2005-2016, The RoboFab Developers: + +- Erik van Blokland +- Tal Leming +- Just van Rossum """ # Restrictions are taken mostly from @@ -93,53 +109,69 @@ class NameTranslationError(Exception): def userNameToFileName(userName: str, existing=(), prefix="", suffix=""): - """ - `existing` should be a set-like object. - - >>> userNameToFileName("a") == "a" - True - >>> userNameToFileName("A") == "A_" - True - >>> userNameToFileName("AE") == "A_E_" - True - >>> userNameToFileName("Ae") == "A_e" - True - >>> userNameToFileName("ae") == "ae" - True - >>> userNameToFileName("aE") == "aE_" - True - >>> userNameToFileName("a.alt") == "a.alt" - True - >>> userNameToFileName("A.alt") == "A_.alt" - True - >>> userNameToFileName("A.Alt") == "A_.A_lt" - True - >>> userNameToFileName("A.aLt") == "A_.aL_t" - True - >>> userNameToFileName(u"A.alT") == "A_.alT_" - True - >>> userNameToFileName("T_H") == "T__H_" - True - >>> userNameToFileName("T_h") == "T__h" - True - >>> userNameToFileName("t_h") == "t_h" - True - >>> userNameToFileName("F_F_I") == "F__F__I_" - True - >>> userNameToFileName("f_f_i") == "f_f_i" - True - >>> userNameToFileName("Aacute_V.swash") == "A_acute_V_.swash" - True - >>> userNameToFileName(".notdef") == "_notdef" - True - >>> userNameToFileName("con") == "_con" - True - >>> userNameToFileName("CON") == "C_O_N_" - True - >>> userNameToFileName("con.alt") == "_con.alt" - True - >>> userNameToFileName("alt.con") == "alt._con" - True + """Converts from a user name to a file name. + + Takes care to avoid illegal characters, reserved file names, ambiguity between + upper- and lower-case characters, and clashes with existing files. + + Args: + userName (str): The input file name. + existing: A case-insensitive list of all existing file names. + prefix: Prefix to be prepended to the file name. + suffix: Suffix to be appended to the file name. + + Returns: + A suitable filename. + + Raises: + NameTranslationError: If no suitable name could be generated. + + Examples:: + + >>> userNameToFileName("a") == "a" + True + >>> userNameToFileName("A") == "A_" + True + >>> userNameToFileName("AE") == "A_E_" + True + >>> userNameToFileName("Ae") == "A_e" + True + >>> userNameToFileName("ae") == "ae" + True + >>> userNameToFileName("aE") == "aE_" + True + >>> userNameToFileName("a.alt") == "a.alt" + True + >>> userNameToFileName("A.alt") == "A_.alt" + True + >>> userNameToFileName("A.Alt") == "A_.A_lt" + True + >>> userNameToFileName("A.aLt") == "A_.aL_t" + True + >>> userNameToFileName(u"A.alT") == "A_.alT_" + True + >>> userNameToFileName("T_H") == "T__H_" + True + >>> userNameToFileName("T_h") == "T__h" + True + >>> userNameToFileName("t_h") == "t_h" + True + >>> userNameToFileName("F_F_I") == "F__F__I_" + True + >>> userNameToFileName("f_f_i") == "f_f_i" + True + >>> userNameToFileName("Aacute_V.swash") == "A_acute_V_.swash" + True + >>> userNameToFileName(".notdef") == "_notdef" + True + >>> userNameToFileName("con") == "_con" + True + >>> userNameToFileName("CON") == "C_O_N_" + True + >>> userNameToFileName("con.alt") == "_con.alt" + True + >>> userNameToFileName("alt.con") == "alt._con" + True """ # the incoming name must be a string if not isinstance(userName, str): @@ -181,33 +213,42 @@ def userNameToFileName(userName: str, existing=(), prefix="", suffix=""): def handleClash1(userName, existing=[], prefix="", suffix=""): - """ - existing should be a case-insensitive list - of all existing file names. - - >>> prefix = ("0" * 5) + "." - >>> suffix = "." + ("0" * 10) - >>> existing = ["a" * 5] - - >>> e = list(existing) - >>> handleClash1(userName="A" * 5, existing=e, - ... prefix=prefix, suffix=suffix) == ( - ... '00000.AAAAA000000000000001.0000000000') - True - - >>> e = list(existing) - >>> e.append(prefix + "aaaaa" + "1".zfill(15) + suffix) - >>> handleClash1(userName="A" * 5, existing=e, - ... prefix=prefix, suffix=suffix) == ( - ... '00000.AAAAA000000000000002.0000000000') - True - - >>> e = list(existing) - >>> e.append(prefix + "AAAAA" + "2".zfill(15) + suffix) - >>> handleClash1(userName="A" * 5, existing=e, - ... prefix=prefix, suffix=suffix) == ( - ... '00000.AAAAA000000000000001.0000000000') - True + """A helper function that resolves collisions with existing names when choosing a filename. + + This function attempts to append an unused integer counter to the filename. + + Args: + userName (str): The input file name. + existing: A case-insensitive list of all existing file names. + prefix: Prefix to be prepended to the file name. + suffix: Suffix to be appended to the file name. + + Returns: + A suitable filename. + + >>> prefix = ("0" * 5) + "." + >>> suffix = "." + ("0" * 10) + >>> existing = ["a" * 5] + + >>> e = list(existing) + >>> handleClash1(userName="A" * 5, existing=e, + ... prefix=prefix, suffix=suffix) == ( + ... '00000.AAAAA000000000000001.0000000000') + True + + >>> e = list(existing) + >>> e.append(prefix + "aaaaa" + "1".zfill(15) + suffix) + >>> handleClash1(userName="A" * 5, existing=e, + ... prefix=prefix, suffix=suffix) == ( + ... '00000.AAAAA000000000000002.0000000000') + True + + >>> e = list(existing) + >>> e.append(prefix + "AAAAA" + "2".zfill(15) + suffix) + >>> handleClash1(userName="A" * 5, existing=e, + ... prefix=prefix, suffix=suffix) == ( + ... '00000.AAAAA000000000000001.0000000000') + True """ # if the prefix length + user name length + suffix length + 15 is at # or past the maximum length, silce 15 characters off of the user name @@ -238,30 +279,44 @@ def handleClash1(userName, existing=[], prefix="", suffix=""): def handleClash2(existing=[], prefix="", suffix=""): - """ - existing should be a case-insensitive list - of all existing file names. - - >>> prefix = ("0" * 5) + "." - >>> suffix = "." + ("0" * 10) - >>> existing = [prefix + str(i) + suffix for i in range(100)] - - >>> e = list(existing) - >>> handleClash2(existing=e, prefix=prefix, suffix=suffix) == ( - ... '00000.100.0000000000') - True - - >>> e = list(existing) - >>> e.remove(prefix + "1" + suffix) - >>> handleClash2(existing=e, prefix=prefix, suffix=suffix) == ( - ... '00000.1.0000000000') - True - - >>> e = list(existing) - >>> e.remove(prefix + "2" + suffix) - >>> handleClash2(existing=e, prefix=prefix, suffix=suffix) == ( - ... '00000.2.0000000000') - True + """A helper function that resolves collisions with existing names when choosing a filename. + + This function is a fallback to :func:`handleClash1`. It attempts to append an unused integer counter to the filename. + + Args: + userName (str): The input file name. + existing: A case-insensitive list of all existing file names. + prefix: Prefix to be prepended to the file name. + suffix: Suffix to be appended to the file name. + + Returns: + A suitable filename. + + Raises: + NameTranslationError: If no suitable name could be generated. + + Examples:: + + >>> prefix = ("0" * 5) + "." + >>> suffix = "." + ("0" * 10) + >>> existing = [prefix + str(i) + suffix for i in range(100)] + + >>> e = list(existing) + >>> handleClash2(existing=e, prefix=prefix, suffix=suffix) == ( + ... '00000.100.0000000000') + True + + >>> e = list(existing) + >>> e.remove(prefix + "1" + suffix) + >>> handleClash2(existing=e, prefix=prefix, suffix=suffix) == ( + ... '00000.1.0000000000') + True + + >>> e = list(existing) + >>> e.remove(prefix + "2" + suffix) + >>> handleClash2(existing=e, prefix=prefix, suffix=suffix) == ( + ... '00000.2.0000000000') + True """ # calculate the longest possible string maxLength = maxFileNameLength - len(prefix) - len(suffix) diff --git a/contrib/python/fonttools/fontTools/ufoLib/glifLib.py b/contrib/python/fonttools/fontTools/ufoLib/glifLib.py index abbda491463..a5a05003ee8 100644 --- a/contrib/python/fonttools/fontTools/ufoLib/glifLib.py +++ b/contrib/python/fonttools/fontTools/ufoLib/glifLib.py @@ -1,11 +1,11 @@ """ -glifLib.py -- Generic module for reading and writing the .glif format. +Generic module for reading and writing the .glif format. More info about the .glif format (GLyphInterchangeFormat) can be found here: http://unifiedfontobject.org -The main class in this module is GlyphSet. It manages a set of .glif files +The main class in this module is :class:`GlyphSet`. It manages a set of .glif files in a folder. It offers two ways to read glyph data, and one way to write glyph data. See the class doc string for details. """ @@ -60,6 +60,13 @@ LAYERINFO_FILENAME = "layerinfo.plist" class GLIFFormatVersion(tuple, _VersionTupleEnumMixin, enum.Enum): + """Class representing the versions of the .glif format supported by the UFO version in use. + + For a given :mod:`fontTools.ufoLib.UFOFormatVersion`, the :func:`supported_versions` method will + return the supported versions of the GLIF file format. If the UFO version is unspecified, the + :func:`supported_versions` method will return all available GLIF format versions. + """ + FORMAT_1_0 = (1, 0) FORMAT_2_0 = (2, 0) diff --git a/contrib/python/fonttools/fontTools/ufoLib/kerning.py b/contrib/python/fonttools/fontTools/ufoLib/kerning.py index 8a1dca5b680..5c84dd720af 100644 --- a/contrib/python/fonttools/fontTools/ufoLib/kerning.py +++ b/contrib/python/fonttools/fontTools/ufoLib/kerning.py @@ -1,43 +1,73 @@ def lookupKerningValue( pair, kerning, groups, fallback=0, glyphToFirstGroup=None, glyphToSecondGroup=None ): - """ - Note: This expects kerning to be a flat dictionary - of kerning pairs, not the nested structure used - in kerning.plist. + """Retrieve the kerning value (if any) between a pair of elements. + + The elments can be either individual glyphs (by name) or kerning + groups (by name), or any combination of the two. + + Args: + pair: + A tuple, in logical order (first, second) with respect + to the reading direction, to query the font for kerning + information on. Each element in the tuple can be either + a glyph name or a kerning group name. + kerning: + A dictionary of kerning pairs. + groups: + A set of kerning groups. + fallback: + The fallback value to return if no kern is found between + the elements in ``pair``. Defaults to 0. + glyphToFirstGroup: + A dictionary mapping glyph names to the first-glyph kerning + groups to which they belong. Defaults to ``None``. + glyphToSecondGroup: + A dictionary mapping glyph names to the second-glyph kerning + groups to which they belong. Defaults to ``None``. + + Returns: + The kerning value between the element pair. If no kerning for + the pair is found, the fallback value is returned. + + Note: This function expects the ``kerning`` argument to be a flat + dictionary of kerning pairs, not the nested structure used in a + kerning.plist file. + + Examples:: - >>> groups = { - ... "public.kern1.O" : ["O", "D", "Q"], - ... "public.kern2.E" : ["E", "F"] - ... } - >>> kerning = { - ... ("public.kern1.O", "public.kern2.E") : -100, - ... ("public.kern1.O", "F") : -200, - ... ("D", "F") : -300 - ... } - >>> lookupKerningValue(("D", "F"), kerning, groups) - -300 - >>> lookupKerningValue(("O", "F"), kerning, groups) - -200 - >>> lookupKerningValue(("O", "E"), kerning, groups) - -100 - >>> lookupKerningValue(("O", "O"), kerning, groups) - 0 - >>> lookupKerningValue(("E", "E"), kerning, groups) - 0 - >>> lookupKerningValue(("E", "O"), kerning, groups) - 0 - >>> lookupKerningValue(("X", "X"), kerning, groups) - 0 - >>> lookupKerningValue(("public.kern1.O", "public.kern2.E"), - ... kerning, groups) - -100 - >>> lookupKerningValue(("public.kern1.O", "F"), kerning, groups) - -200 - >>> lookupKerningValue(("O", "public.kern2.E"), kerning, groups) - -100 - >>> lookupKerningValue(("public.kern1.X", "public.kern2.X"), kerning, groups) - 0 + >>> groups = { + ... "public.kern1.O" : ["O", "D", "Q"], + ... "public.kern2.E" : ["E", "F"] + ... } + >>> kerning = { + ... ("public.kern1.O", "public.kern2.E") : -100, + ... ("public.kern1.O", "F") : -200, + ... ("D", "F") : -300 + ... } + >>> lookupKerningValue(("D", "F"), kerning, groups) + -300 + >>> lookupKerningValue(("O", "F"), kerning, groups) + -200 + >>> lookupKerningValue(("O", "E"), kerning, groups) + -100 + >>> lookupKerningValue(("O", "O"), kerning, groups) + 0 + >>> lookupKerningValue(("E", "E"), kerning, groups) + 0 + >>> lookupKerningValue(("E", "O"), kerning, groups) + 0 + >>> lookupKerningValue(("X", "X"), kerning, groups) + 0 + >>> lookupKerningValue(("public.kern1.O", "public.kern2.E"), + ... kerning, groups) + -100 + >>> lookupKerningValue(("public.kern1.O", "F"), kerning, groups) + -200 + >>> lookupKerningValue(("O", "public.kern2.E"), kerning, groups) + -100 + >>> lookupKerningValue(("public.kern1.X", "public.kern2.X"), kerning, groups) + 0 """ # quickly check to see if the pair is in the kerning dictionary if pair in kerning: diff --git a/contrib/python/fonttools/fontTools/ufoLib/utils.py b/contrib/python/fonttools/fontTools/ufoLib/utils.py index 45ec1c564b7..45ae5e812eb 100644 --- a/contrib/python/fonttools/fontTools/ufoLib/utils.py +++ b/contrib/python/fonttools/fontTools/ufoLib/utils.py @@ -1,5 +1,8 @@ -"""The module contains miscellaneous helpers. -It's not considered part of the public ufoLib API. +"""This module contains miscellaneous helpers. + +It is not considered part of the public ufoLib API. It does, however, +define the :py:obj:`.deprecated` decorator that is used elsewhere in +the module. """ import warnings diff --git a/contrib/python/fonttools/fontTools/unicodedata/Mirrored.py b/contrib/python/fonttools/fontTools/unicodedata/Mirrored.py new file mode 100644 index 00000000000..75b51a90123 --- /dev/null +++ b/contrib/python/fonttools/fontTools/unicodedata/Mirrored.py @@ -0,0 +1,446 @@ +# -*- coding: utf-8 -*- +# +# NOTE: The mappings in this file were generated from the command line: +# cat BidiMirroring.txt | grep "^[0-9A-F]" | sed "s/;//" | awk '{print " 0x"$1": 0x"$2","}' +# +# Source: http://www.unicode.org/Public/UNIDATA/BidiMirroring.txt +# License: http://unicode.org/copyright.html#License +# +# BidiMirroring-16.0.0.txt +# Date: 2024-01-30 +# © 2024 Unicode®, Inc. +# Unicode and the Unicode Logo are registered trademarks of Unicode, Inc. in the U.S. and other countries. +# For terms of use and license, see https://www.unicode.org/terms_of_use.html +# +# Unicode Character Database +# For documentation, see https://www.unicode.org/reports/tr44/ +MIRRORED = { + 0x0028: 0x0029, + 0x0029: 0x0028, + 0x003C: 0x003E, + 0x003E: 0x003C, + 0x005B: 0x005D, + 0x005D: 0x005B, + 0x007B: 0x007D, + 0x007D: 0x007B, + 0x00AB: 0x00BB, + 0x00BB: 0x00AB, + 0x0F3A: 0x0F3B, + 0x0F3B: 0x0F3A, + 0x0F3C: 0x0F3D, + 0x0F3D: 0x0F3C, + 0x169B: 0x169C, + 0x169C: 0x169B, + 0x2039: 0x203A, + 0x203A: 0x2039, + 0x2045: 0x2046, + 0x2046: 0x2045, + 0x207D: 0x207E, + 0x207E: 0x207D, + 0x208D: 0x208E, + 0x208E: 0x208D, + 0x2208: 0x220B, + 0x2209: 0x220C, + 0x220A: 0x220D, + 0x220B: 0x2208, + 0x220C: 0x2209, + 0x220D: 0x220A, + 0x2215: 0x29F5, + 0x221F: 0x2BFE, + 0x2220: 0x29A3, + 0x2221: 0x299B, + 0x2222: 0x29A0, + 0x2224: 0x2AEE, + 0x223C: 0x223D, + 0x223D: 0x223C, + 0x2243: 0x22CD, + 0x2245: 0x224C, + 0x224C: 0x2245, + 0x2252: 0x2253, + 0x2253: 0x2252, + 0x2254: 0x2255, + 0x2255: 0x2254, + 0x2264: 0x2265, + 0x2265: 0x2264, + 0x2266: 0x2267, + 0x2267: 0x2266, + 0x2268: 0x2269, + 0x2269: 0x2268, + 0x226A: 0x226B, + 0x226B: 0x226A, + 0x226E: 0x226F, + 0x226F: 0x226E, + 0x2270: 0x2271, + 0x2271: 0x2270, + 0x2272: 0x2273, + 0x2273: 0x2272, + 0x2274: 0x2275, + 0x2275: 0x2274, + 0x2276: 0x2277, + 0x2277: 0x2276, + 0x2278: 0x2279, + 0x2279: 0x2278, + 0x227A: 0x227B, + 0x227B: 0x227A, + 0x227C: 0x227D, + 0x227D: 0x227C, + 0x227E: 0x227F, + 0x227F: 0x227E, + 0x2280: 0x2281, + 0x2281: 0x2280, + 0x2282: 0x2283, + 0x2283: 0x2282, + 0x2284: 0x2285, + 0x2285: 0x2284, + 0x2286: 0x2287, + 0x2287: 0x2286, + 0x2288: 0x2289, + 0x2289: 0x2288, + 0x228A: 0x228B, + 0x228B: 0x228A, + 0x228F: 0x2290, + 0x2290: 0x228F, + 0x2291: 0x2292, + 0x2292: 0x2291, + 0x2298: 0x29B8, + 0x22A2: 0x22A3, + 0x22A3: 0x22A2, + 0x22A6: 0x2ADE, + 0x22A8: 0x2AE4, + 0x22A9: 0x2AE3, + 0x22AB: 0x2AE5, + 0x22B0: 0x22B1, + 0x22B1: 0x22B0, + 0x22B2: 0x22B3, + 0x22B3: 0x22B2, + 0x22B4: 0x22B5, + 0x22B5: 0x22B4, + 0x22B6: 0x22B7, + 0x22B7: 0x22B6, + 0x22B8: 0x27DC, + 0x22C9: 0x22CA, + 0x22CA: 0x22C9, + 0x22CB: 0x22CC, + 0x22CC: 0x22CB, + 0x22CD: 0x2243, + 0x22D0: 0x22D1, + 0x22D1: 0x22D0, + 0x22D6: 0x22D7, + 0x22D7: 0x22D6, + 0x22D8: 0x22D9, + 0x22D9: 0x22D8, + 0x22DA: 0x22DB, + 0x22DB: 0x22DA, + 0x22DC: 0x22DD, + 0x22DD: 0x22DC, + 0x22DE: 0x22DF, + 0x22DF: 0x22DE, + 0x22E0: 0x22E1, + 0x22E1: 0x22E0, + 0x22E2: 0x22E3, + 0x22E3: 0x22E2, + 0x22E4: 0x22E5, + 0x22E5: 0x22E4, + 0x22E6: 0x22E7, + 0x22E7: 0x22E6, + 0x22E8: 0x22E9, + 0x22E9: 0x22E8, + 0x22EA: 0x22EB, + 0x22EB: 0x22EA, + 0x22EC: 0x22ED, + 0x22ED: 0x22EC, + 0x22F0: 0x22F1, + 0x22F1: 0x22F0, + 0x22F2: 0x22FA, + 0x22F3: 0x22FB, + 0x22F4: 0x22FC, + 0x22F6: 0x22FD, + 0x22F7: 0x22FE, + 0x22FA: 0x22F2, + 0x22FB: 0x22F3, + 0x22FC: 0x22F4, + 0x22FD: 0x22F6, + 0x22FE: 0x22F7, + 0x2308: 0x2309, + 0x2309: 0x2308, + 0x230A: 0x230B, + 0x230B: 0x230A, + 0x2329: 0x232A, + 0x232A: 0x2329, + 0x2768: 0x2769, + 0x2769: 0x2768, + 0x276A: 0x276B, + 0x276B: 0x276A, + 0x276C: 0x276D, + 0x276D: 0x276C, + 0x276E: 0x276F, + 0x276F: 0x276E, + 0x2770: 0x2771, + 0x2771: 0x2770, + 0x2772: 0x2773, + 0x2773: 0x2772, + 0x2774: 0x2775, + 0x2775: 0x2774, + 0x27C3: 0x27C4, + 0x27C4: 0x27C3, + 0x27C5: 0x27C6, + 0x27C6: 0x27C5, + 0x27C8: 0x27C9, + 0x27C9: 0x27C8, + 0x27CB: 0x27CD, + 0x27CD: 0x27CB, + 0x27D5: 0x27D6, + 0x27D6: 0x27D5, + 0x27DC: 0x22B8, + 0x27DD: 0x27DE, + 0x27DE: 0x27DD, + 0x27E2: 0x27E3, + 0x27E3: 0x27E2, + 0x27E4: 0x27E5, + 0x27E5: 0x27E4, + 0x27E6: 0x27E7, + 0x27E7: 0x27E6, + 0x27E8: 0x27E9, + 0x27E9: 0x27E8, + 0x27EA: 0x27EB, + 0x27EB: 0x27EA, + 0x27EC: 0x27ED, + 0x27ED: 0x27EC, + 0x27EE: 0x27EF, + 0x27EF: 0x27EE, + 0x2983: 0x2984, + 0x2984: 0x2983, + 0x2985: 0x2986, + 0x2986: 0x2985, + 0x2987: 0x2988, + 0x2988: 0x2987, + 0x2989: 0x298A, + 0x298A: 0x2989, + 0x298B: 0x298C, + 0x298C: 0x298B, + 0x298D: 0x2990, + 0x298E: 0x298F, + 0x298F: 0x298E, + 0x2990: 0x298D, + 0x2991: 0x2992, + 0x2992: 0x2991, + 0x2993: 0x2994, + 0x2994: 0x2993, + 0x2995: 0x2996, + 0x2996: 0x2995, + 0x2997: 0x2998, + 0x2998: 0x2997, + 0x299B: 0x2221, + 0x29A0: 0x2222, + 0x29A3: 0x2220, + 0x29A4: 0x29A5, + 0x29A5: 0x29A4, + 0x29A8: 0x29A9, + 0x29A9: 0x29A8, + 0x29AA: 0x29AB, + 0x29AB: 0x29AA, + 0x29AC: 0x29AD, + 0x29AD: 0x29AC, + 0x29AE: 0x29AF, + 0x29AF: 0x29AE, + 0x29B8: 0x2298, + 0x29C0: 0x29C1, + 0x29C1: 0x29C0, + 0x29C4: 0x29C5, + 0x29C5: 0x29C4, + 0x29CF: 0x29D0, + 0x29D0: 0x29CF, + 0x29D1: 0x29D2, + 0x29D2: 0x29D1, + 0x29D4: 0x29D5, + 0x29D5: 0x29D4, + 0x29D8: 0x29D9, + 0x29D9: 0x29D8, + 0x29DA: 0x29DB, + 0x29DB: 0x29DA, + 0x29E8: 0x29E9, + 0x29E9: 0x29E8, + 0x29F5: 0x2215, + 0x29F8: 0x29F9, + 0x29F9: 0x29F8, + 0x29FC: 0x29FD, + 0x29FD: 0x29FC, + 0x2A2B: 0x2A2C, + 0x2A2C: 0x2A2B, + 0x2A2D: 0x2A2E, + 0x2A2E: 0x2A2D, + 0x2A34: 0x2A35, + 0x2A35: 0x2A34, + 0x2A3C: 0x2A3D, + 0x2A3D: 0x2A3C, + 0x2A64: 0x2A65, + 0x2A65: 0x2A64, + 0x2A79: 0x2A7A, + 0x2A7A: 0x2A79, + 0x2A7B: 0x2A7C, + 0x2A7C: 0x2A7B, + 0x2A7D: 0x2A7E, + 0x2A7E: 0x2A7D, + 0x2A7F: 0x2A80, + 0x2A80: 0x2A7F, + 0x2A81: 0x2A82, + 0x2A82: 0x2A81, + 0x2A83: 0x2A84, + 0x2A84: 0x2A83, + 0x2A85: 0x2A86, + 0x2A86: 0x2A85, + 0x2A87: 0x2A88, + 0x2A88: 0x2A87, + 0x2A89: 0x2A8A, + 0x2A8A: 0x2A89, + 0x2A8B: 0x2A8C, + 0x2A8C: 0x2A8B, + 0x2A8D: 0x2A8E, + 0x2A8E: 0x2A8D, + 0x2A8F: 0x2A90, + 0x2A90: 0x2A8F, + 0x2A91: 0x2A92, + 0x2A92: 0x2A91, + 0x2A93: 0x2A94, + 0x2A94: 0x2A93, + 0x2A95: 0x2A96, + 0x2A96: 0x2A95, + 0x2A97: 0x2A98, + 0x2A98: 0x2A97, + 0x2A99: 0x2A9A, + 0x2A9A: 0x2A99, + 0x2A9B: 0x2A9C, + 0x2A9C: 0x2A9B, + 0x2A9D: 0x2A9E, + 0x2A9E: 0x2A9D, + 0x2A9F: 0x2AA0, + 0x2AA0: 0x2A9F, + 0x2AA1: 0x2AA2, + 0x2AA2: 0x2AA1, + 0x2AA6: 0x2AA7, + 0x2AA7: 0x2AA6, + 0x2AA8: 0x2AA9, + 0x2AA9: 0x2AA8, + 0x2AAA: 0x2AAB, + 0x2AAB: 0x2AAA, + 0x2AAC: 0x2AAD, + 0x2AAD: 0x2AAC, + 0x2AAF: 0x2AB0, + 0x2AB0: 0x2AAF, + 0x2AB1: 0x2AB2, + 0x2AB2: 0x2AB1, + 0x2AB3: 0x2AB4, + 0x2AB4: 0x2AB3, + 0x2AB5: 0x2AB6, + 0x2AB6: 0x2AB5, + 0x2AB7: 0x2AB8, + 0x2AB8: 0x2AB7, + 0x2AB9: 0x2ABA, + 0x2ABA: 0x2AB9, + 0x2ABB: 0x2ABC, + 0x2ABC: 0x2ABB, + 0x2ABD: 0x2ABE, + 0x2ABE: 0x2ABD, + 0x2ABF: 0x2AC0, + 0x2AC0: 0x2ABF, + 0x2AC1: 0x2AC2, + 0x2AC2: 0x2AC1, + 0x2AC3: 0x2AC4, + 0x2AC4: 0x2AC3, + 0x2AC5: 0x2AC6, + 0x2AC6: 0x2AC5, + 0x2AC7: 0x2AC8, + 0x2AC8: 0x2AC7, + 0x2AC9: 0x2ACA, + 0x2ACA: 0x2AC9, + 0x2ACB: 0x2ACC, + 0x2ACC: 0x2ACB, + 0x2ACD: 0x2ACE, + 0x2ACE: 0x2ACD, + 0x2ACF: 0x2AD0, + 0x2AD0: 0x2ACF, + 0x2AD1: 0x2AD2, + 0x2AD2: 0x2AD1, + 0x2AD3: 0x2AD4, + 0x2AD4: 0x2AD3, + 0x2AD5: 0x2AD6, + 0x2AD6: 0x2AD5, + 0x2ADE: 0x22A6, + 0x2AE3: 0x22A9, + 0x2AE4: 0x22A8, + 0x2AE5: 0x22AB, + 0x2AEC: 0x2AED, + 0x2AED: 0x2AEC, + 0x2AEE: 0x2224, + 0x2AF7: 0x2AF8, + 0x2AF8: 0x2AF7, + 0x2AF9: 0x2AFA, + 0x2AFA: 0x2AF9, + 0x2BFE: 0x221F, + 0x2E02: 0x2E03, + 0x2E03: 0x2E02, + 0x2E04: 0x2E05, + 0x2E05: 0x2E04, + 0x2E09: 0x2E0A, + 0x2E0A: 0x2E09, + 0x2E0C: 0x2E0D, + 0x2E0D: 0x2E0C, + 0x2E1C: 0x2E1D, + 0x2E1D: 0x2E1C, + 0x2E20: 0x2E21, + 0x2E21: 0x2E20, + 0x2E22: 0x2E23, + 0x2E23: 0x2E22, + 0x2E24: 0x2E25, + 0x2E25: 0x2E24, + 0x2E26: 0x2E27, + 0x2E27: 0x2E26, + 0x2E28: 0x2E29, + 0x2E29: 0x2E28, + 0x2E55: 0x2E56, + 0x2E56: 0x2E55, + 0x2E57: 0x2E58, + 0x2E58: 0x2E57, + 0x2E59: 0x2E5A, + 0x2E5A: 0x2E59, + 0x2E5B: 0x2E5C, + 0x2E5C: 0x2E5B, + 0x3008: 0x3009, + 0x3009: 0x3008, + 0x300A: 0x300B, + 0x300B: 0x300A, + 0x300C: 0x300D, + 0x300D: 0x300C, + 0x300E: 0x300F, + 0x300F: 0x300E, + 0x3010: 0x3011, + 0x3011: 0x3010, + 0x3014: 0x3015, + 0x3015: 0x3014, + 0x3016: 0x3017, + 0x3017: 0x3016, + 0x3018: 0x3019, + 0x3019: 0x3018, + 0x301A: 0x301B, + 0x301B: 0x301A, + 0xFE59: 0xFE5A, + 0xFE5A: 0xFE59, + 0xFE5B: 0xFE5C, + 0xFE5C: 0xFE5B, + 0xFE5D: 0xFE5E, + 0xFE5E: 0xFE5D, + 0xFE64: 0xFE65, + 0xFE65: 0xFE64, + 0xFF08: 0xFF09, + 0xFF09: 0xFF08, + 0xFF1C: 0xFF1E, + 0xFF1E: 0xFF1C, + 0xFF3B: 0xFF3D, + 0xFF3D: 0xFF3B, + 0xFF5B: 0xFF5D, + 0xFF5D: 0xFF5B, + 0xFF5F: 0xFF60, + 0xFF60: 0xFF5F, + 0xFF62: 0xFF63, + 0xFF63: 0xFF62, +} diff --git a/contrib/python/fonttools/fontTools/unicodedata/__init__.py b/contrib/python/fonttools/fontTools/unicodedata/__init__.py index edae44ec718..1adb07d2896 100644 --- a/contrib/python/fonttools/fontTools/unicodedata/__init__.py +++ b/contrib/python/fonttools/fontTools/unicodedata/__init__.py @@ -15,8 +15,7 @@ except ImportError: # pragma: no cover # fall back to built-in unicodedata (possibly outdated) from unicodedata import * -from . import Blocks, Scripts, ScriptExtensions, OTTags - +from . import Blocks, Mirrored, Scripts, ScriptExtensions, OTTags __all__ = [ # names from built-in unicodedata module @@ -46,6 +45,11 @@ __all__ = [ ] +def mirrored(code): + """If code (unicode codepoint) has a mirrored version returns it, otherwise None.""" + return Mirrored.MIRRORED.get(code) + + def script(char): """Return the four-letter script code assigned to the Unicode character 'char' as string. diff --git a/contrib/python/fonttools/fontTools/varLib/__init__.py b/contrib/python/fonttools/fontTools/varLib/__init__.py index f3d8d59dc5a..e3c00d73fa9 100644 --- a/contrib/python/fonttools/fontTools/varLib/__init__.py +++ b/contrib/python/fonttools/fontTools/varLib/__init__.py @@ -555,6 +555,8 @@ def _add_VHVAR(font, axisTags, tableFields, getAdvanceMetrics): varData.addItem(vhAdvanceDeltasAndSupports[glyphName][0], round=noRound) varData.optimize() directStore = builder.buildVarStore(varTupleList, [varData]) + # remove unused regions from VarRegionList + directStore.prune_regions() # Build optimized indirect mapping storeBuilder = varStore.OnlineVarStoreBuilder(axisTags) diff --git a/contrib/python/fonttools/fontTools/voltLib/__main__.py b/contrib/python/fonttools/fontTools/voltLib/__main__.py new file mode 100644 index 00000000000..aa2c3b3e610 --- /dev/null +++ b/contrib/python/fonttools/fontTools/voltLib/__main__.py @@ -0,0 +1,206 @@ +import argparse +import logging +import sys +from io import StringIO +from pathlib import Path + +from fontTools import configLogger +from fontTools.feaLib.builder import addOpenTypeFeaturesFromString +from fontTools.feaLib.error import FeatureLibError +from fontTools.feaLib.lexer import Lexer +from fontTools.misc.cliTools import makeOutputFileName +from fontTools.ttLib import TTFont, TTLibError +from fontTools.voltLib.parser import Parser +from fontTools.voltLib.voltToFea import TABLES, VoltToFea + +log = logging.getLogger("fontTools.feaLib") + +SUPPORTED_TABLES = TABLES + ["cmap"] + + +def invalid_fea_glyph_name(name): + """Check if the glyph name is valid according to FEA syntax.""" + if name[0] not in Lexer.CHAR_NAME_START_: + return True + if any(c not in Lexer.CHAR_NAME_CONTINUATION_ for c in name[1:]): + return True + return False + + +def sanitize_glyph_name(name): + """Sanitize the glyph name to ensure it is valid according to FEA syntax.""" + sanitized = "" + for i, c in enumerate(name): + if i == 0 and c not in Lexer.CHAR_NAME_START_: + sanitized += "a" + c + elif c not in Lexer.CHAR_NAME_CONTINUATION_: + sanitized += "_" + else: + sanitized += c + + return sanitized + + +def main(args=None): + """Build tables from a MS VOLT project into an OTF font""" + parser = argparse.ArgumentParser( + description="Use fontTools to compile MS VOLT projects." + ) + parser.add_argument( + "input", + metavar="INPUT", + help="Path to the input font/VTP file to process", + type=Path, + ) + parser.add_argument( + "-f", + "--font", + metavar="INPUT_FONT", + help="Path to the input font (if INPUT is a VTP file)", + type=Path, + ) + parser.add_argument( + "-o", + "--output", + dest="output", + metavar="OUTPUT", + help="Path to the output font.", + type=Path, + ) + parser.add_argument( + "-t", + "--tables", + metavar="TABLE_TAG", + choices=SUPPORTED_TABLES, + nargs="+", + help="Specify the table(s) to be built.", + ) + parser.add_argument( + "-F", + "--debug-feature-file", + help="Write the generated feature file to disk.", + action="store_true", + ) + parser.add_argument( + "--ship", + help="Remove source VOLT tables from output font.", + action="store_true", + ) + parser.add_argument( + "-v", + "--verbose", + help="Increase the logger verbosity. Multiple -v options are allowed.", + action="count", + default=0, + ) + parser.add_argument( + "-T", + "--traceback", + help="show traceback for exceptions.", + action="store_true", + ) + options = parser.parse_args(args) + + levels = ["WARNING", "INFO", "DEBUG"] + configLogger(level=levels[min(len(levels) - 1, options.verbose)]) + + output_font = options.output or Path( + makeOutputFileName(options.font or options.input) + ) + log.info(f"Compiling MS VOLT to '{output_font}'") + + file_or_path = options.input + font = None + + # If the input is a font file, extract the VOLT data from the "TSIV" table + try: + font = TTFont(file_or_path) + if "TSIV" in font: + file_or_path = StringIO(font["TSIV"].data.decode("utf-8")) + else: + log.error('"TSIV" table is missing') + return 1 + except TTLibError: + pass + + # If input is not a font file, the font must be provided + if font is None: + if not options.font: + log.error("Please provide an input font") + return 1 + font = TTFont(options.font) + + # FEA syntax does not allow some glyph names that VOLT accepts, so if we + # found such glyph name we will temporarily rename such glyphs. + glyphOrder = font.getGlyphOrder() + tempGlyphOrder = None + if any(invalid_fea_glyph_name(n) for n in glyphOrder): + tempGlyphOrder = [] + for n in glyphOrder: + if invalid_fea_glyph_name(n): + n = sanitize_glyph_name(n) + existing = set(tempGlyphOrder) | set(glyphOrder) + while n in existing: + n = "a" + n + tempGlyphOrder.append(n) + font.setGlyphOrder(tempGlyphOrder) + + doc = Parser(file_or_path).parse() + + log.info("Converting VTP data to FEA") + converter = VoltToFea(doc, font) + try: + fea = converter.convert(options.tables, ignore_unsupported_settings=True) + except NotImplementedError as e: + if options.traceback: + raise + location = getattr(e.args[0], "location", None) + message = f'"{e}" is not supported' + if location: + path, line, column = location + log.error(f"{path}:{line}:{column}: {message}") + else: + log.error(message) + return 1 + + fea_filename = options.input + if options.debug_feature_file: + fea_filename = output_font.with_suffix(".fea") + log.info(f"Writing FEA to '{fea_filename}'") + with open(fea_filename, "w") as fp: + fp.write(fea) + + log.info("Compiling FEA to OpenType tables") + try: + addOpenTypeFeaturesFromString( + font, + fea, + filename=fea_filename, + tables=options.tables, + ) + except FeatureLibError as e: + if options.traceback: + raise + log.error(e) + return 1 + + if options.ship: + for tag in ["TSIV", "TSIS", "TSIP", "TSID"]: + if tag in font: + del font[tag] + + # Restore original glyph names. + if tempGlyphOrder: + import io + + f = io.BytesIO() + font.save(f) + font = TTFont(f) + font.setGlyphOrder(glyphOrder) + font["post"].extraNames = [] + + font.save(output_font) + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/contrib/python/fonttools/fontTools/voltLib/ast.py b/contrib/python/fonttools/fontTools/voltLib/ast.py index 82c2cca8b7f..dba8f4d45b3 100644 --- a/contrib/python/fonttools/fontTools/voltLib/ast.py +++ b/contrib/python/fonttools/fontTools/voltLib/ast.py @@ -317,6 +317,10 @@ class SubstitutionLigatureDefinition(SubstitutionDefinition): pass +class SubstitutionAlternateDefinition(SubstitutionDefinition): + pass + + class SubstitutionReverseChainingSingleDefinition(SubstitutionDefinition): pass diff --git a/contrib/python/fonttools/fontTools/voltLib/parser.py b/contrib/python/fonttools/fontTools/voltLib/parser.py index 1fa6b11d027..31a76e69bff 100644 --- a/contrib/python/fonttools/fontTools/voltLib/parser.py +++ b/contrib/python/fonttools/fontTools/voltLib/parser.py @@ -313,19 +313,27 @@ class Parser(object): self.expect_keyword_("END_SUBSTITUTION") max_src = max([len(cov) for cov in src]) max_dest = max([len(cov) for cov in dest]) + # many to many or mixed is invalid - if (max_src > 1 and max_dest > 1) or ( - reversal and (max_src > 1 or max_dest > 1) - ): + if max_src > 1 and max_dest > 1: raise VoltLibError("Invalid substitution type", location) + mapping = dict(zip(tuple(src), tuple(dest))) if max_src == 1 and max_dest == 1: - if reversal: - sub = ast.SubstitutionReverseChainingSingleDefinition( - mapping, location=location - ) + # Alternate substitutions are represented by adding multiple + # substitutions for the same glyph, so we detect that here + glyphs = [x.glyphSet() for cov in src for x in cov] # flatten src + if len(set(glyphs)) != len(glyphs): # src has duplicates + sub = ast.SubstitutionAlternateDefinition(mapping, location=location) else: - sub = ast.SubstitutionSingleDefinition(mapping, location=location) + if reversal: + # Reversal is valid only for single glyph substitutions + # and VOLT ignores it otherwise. + sub = ast.SubstitutionReverseChainingSingleDefinition( + mapping, location=location + ) + else: + sub = ast.SubstitutionSingleDefinition(mapping, location=location) elif max_src == 1 and max_dest > 1: sub = ast.SubstitutionMultipleDefinition(mapping, location=location) elif max_src > 1 and max_dest == 1: diff --git a/contrib/python/fonttools/fontTools/voltLib/voltToFea.py b/contrib/python/fonttools/fontTools/voltLib/voltToFea.py index c77d5ad1111..d552f4b52a8 100644 --- a/contrib/python/fonttools/fontTools/voltLib/voltToFea.py +++ b/contrib/python/fonttools/fontTools/voltLib/voltToFea.py @@ -46,6 +46,7 @@ Limitations import logging import re from io import StringIO +from graphlib import TopologicalSorter from fontTools.feaLib import ast from fontTools.ttLib import TTFont, TTLibError @@ -57,32 +58,39 @@ log = logging.getLogger("fontTools.voltLib.voltToFea") TABLES = ["GDEF", "GSUB", "GPOS"] -class MarkClassDefinition(ast.MarkClassDefinition): - def asFea(self, indent=""): - res = "" - if not getattr(self, "used", False): - res += "#" - res += ast.MarkClassDefinition.asFea(self, indent) - return res - - -# For sorting voltLib.ast.GlyphDefinition, see its use below. -class Group: - def __init__(self, group): - self.name = group.name.lower() - self.groups = [ - x.group.lower() for x in group.enum.enum if isinstance(x, VAst.GroupName) +def _flatten_group(group): + ret = [] + if isinstance(group, (tuple, list)): + for item in group: + ret.extend(_flatten_group(item)) + elif hasattr(group, "enum"): + ret.extend(_flatten_group(group.enum)) + else: + ret.append(group) + return ret + + +# Topologically sort of group definitions to ensure that all groups are defined +# before they are referenced. This is necessary because FEA requires it but +# VOLT does not, see below. +def sort_groups(groups): + group_map = {group.name.lower(): group for group in groups} + graph = { + group.name.lower(): [ + x.group.lower() + for x in _flatten_group(group) + if isinstance(x, VAst.GroupName) ] + for group in groups + } + sorter = TopologicalSorter(graph) + return [group_map[name] for name in sorter.static_order()] + - def __lt__(self, other): - if self.name in other.groups: - return True - if other.name in self.groups: - return False - if self.groups and not other.groups: - return False - if not self.groups and other.groups: - return True +class Lookup(ast.LookupBlock): + def __init__(self, name, use_extension=False, location=None): + super().__init__(name, use_extension, location) + self.chained = [] class VoltToFea: @@ -90,7 +98,10 @@ class VoltToFea: _NOT_CLASS_NAME_RE = re.compile(r"[^A-Za-z_0-9.\-]") def __init__(self, file_or_path, font=None): - self._file_or_path = file_or_path + if isinstance(file_or_path, VAst.VoltFile): + self._doc, self._file_or_path = file_or_path, None + else: + self._doc, self._file_or_path = None, file_or_path self._font = font self._glyph_map = {} @@ -128,23 +139,26 @@ class VoltToFea: self._class_names[name] = res return self._class_names[name] - def _collectStatements(self, doc, tables): + def _collectStatements(self, doc, tables, ignore_unsupported_settings=False): + # Collect glyph difinitions first, as we need them to map VOLT glyph names to font glyph name. + for statement in doc.statements: + if isinstance(statement, VAst.GlyphDefinition): + self._glyphDefinition(statement) + # Collect and sort group definitions first, to make sure a group # definition that references other groups comes after them since VOLT # does not enforce such ordering, and feature file require it. groups = [s for s in doc.statements if isinstance(s, VAst.GroupDefinition)] - for statement in sorted(groups, key=lambda x: Group(x)): - self._groupDefinition(statement) + for group in sort_groups(groups): + self._groupDefinition(group) for statement in doc.statements: - if isinstance(statement, VAst.GlyphDefinition): - self._glyphDefinition(statement) - elif isinstance(statement, VAst.AnchorDefinition): + if isinstance(statement, VAst.AnchorDefinition): if "GPOS" in tables: self._anchorDefinition(statement) elif isinstance(statement, VAst.SettingDefinition): - self._settingDefinition(statement) - elif isinstance(statement, VAst.GroupDefinition): + self._settingDefinition(statement, ignore_unsupported_settings) + elif isinstance(statement, (VAst.GlyphDefinition, VAst.GroupDefinition)): pass # Handled above elif isinstance(statement, VAst.ScriptDefinition): self._scriptDefinition(statement) @@ -176,35 +190,57 @@ class VoltToFea: if self._lookups: statements.append(ast.Comment("\n# Lookups")) for lookup in self._lookups.values(): - statements.extend(getattr(lookup, "targets", [])) + statements.extend(lookup.chained) statements.append(lookup) # Prune features features = self._features.copy() - for ftag in features: - scripts = features[ftag] - for stag in scripts: - langs = scripts[stag] - for ltag in langs: - langs[ltag] = [l for l in langs[ltag] if l.lower() in self._lookups] - scripts[stag] = {t: l for t, l in langs.items() if l} - features[ftag] = {t: s for t, s in scripts.items() if s} + for feature_tag in features: + scripts = features[feature_tag] + for script_tag in scripts: + langs = scripts[script_tag] + for language_tag in langs: + langs[language_tag] = [ + l for l in langs[language_tag] if l.lower() in self._lookups + ] + scripts[script_tag] = {t: l for t, l in langs.items() if l} + features[feature_tag] = {t: s for t, s in scripts.items() if s} features = {t: f for t, f in features.items() if f} if features: statements.append(ast.Comment("# Features")) - for ftag, scripts in features.items(): - feature = ast.FeatureBlock(ftag) - stags = sorted(scripts, key=lambda k: 0 if k == "DFLT" else 1) - for stag in stags: - feature.statements.append(ast.ScriptStatement(stag)) - ltags = sorted(scripts[stag], key=lambda k: 0 if k == "dflt" else 1) - for ltag in ltags: - include_default = True if ltag == "dflt" else False - feature.statements.append( - ast.LanguageStatement(ltag, include_default=include_default) + for feature_tag, scripts in features.items(): + feature = ast.FeatureBlock(feature_tag) + script_tags = sorted(scripts, key=lambda k: 0 if k == "DFLT" else 1) + if feature_tag == "aalt" and len(script_tags) > 1: + log.warning( + "FEA syntax does not allow script statements in 'aalt' feature, " + "so only lookups from the first script will be included." + ) + script_tags = script_tags[:1] + for script_tag in script_tags: + if feature_tag != "aalt": + feature.statements.append(ast.ScriptStatement(script_tag)) + language_tags = sorted( + scripts[script_tag], + key=lambda k: 0 if k == "dflt" else 1, + ) + if feature_tag == "aalt" and len(language_tags) > 1: + log.warning( + "FEA syntax does not allow language statements in 'aalt' feature, " + "so only lookups from the first language will be included." ) - for name in scripts[stag][ltag]: + language_tags = language_tags[:1] + for language_tag in language_tags: + if feature_tag != "aalt": + include_default = True if language_tag == "dflt" else False + feature.statements.append( + ast.LanguageStatement( + language_tag.ljust(4), + include_default=include_default, + ) + ) + for name in scripts[script_tag][language_tag]: lookup = self._lookups[name.lower()] lookupref = ast.LookupReferenceStatement(lookup) feature.statements.append(lookupref) @@ -227,15 +263,17 @@ class VoltToFea: return doc - def convert(self, tables=None): - doc = VoltParser(self._file_or_path).parse() + def convert(self, tables=None, ignore_unsupported_settings=False): + if self._doc is None: + self._doc = VoltParser(self._file_or_path).parse() + doc = self._doc if tables is None: tables = TABLES if self._font is not None: self._glyph_order = self._font.getGlyphOrder() - self._collectStatements(doc, tables) + self._collectStatements(doc, tables, ignore_unsupported_settings) fea = self._buildFeatureFile(tables) return fea.asFea() @@ -253,7 +291,13 @@ class VoltToFea: name = group return ast.GlyphClassName(self._glyphclasses[name.lower()]) - def _coverage(self, coverage): + def _glyphSet(self, item): + return [ + (self._glyphName(x) if isinstance(x, (str, VAst.GlyphName)) else x) + for x in item.glyphSet() + ] + + def _coverage(self, coverage, flatten=False): items = [] for item in coverage: if isinstance(item, VAst.GlyphName): @@ -261,31 +305,38 @@ class VoltToFea: elif isinstance(item, VAst.GroupName): items.append(self._groupName(item)) elif isinstance(item, VAst.Enum): - items.append(self._enum(item)) + item = self._coverage(item.enum, flatten=True) + if flatten: + items.extend(item) + else: + items.append(ast.GlyphClass(item)) elif isinstance(item, VAst.Range): - items.append((item.start, item.end)) + item = self._glyphSet(item) + if flatten: + items.extend(item) + else: + items.append(ast.GlyphClass(item)) else: raise NotImplementedError(item) return items - def _enum(self, enum): - return ast.GlyphClass(self._coverage(enum.enum)) - def _context(self, context): out = [] for item in context: - coverage = self._coverage(item) - if not isinstance(coverage, (tuple, list)): - coverage = [coverage] - out.extend(coverage) + coverage = self._coverage(item, flatten=True) + if len(coverage) > 1: + coverage = ast.GlyphClass(coverage) + else: + coverage = coverage[0] + out.append(coverage) return out def _groupDefinition(self, group): name = self._className(group.name) - glyphs = self._enum(group.enum) - glyphclass = ast.GlyphClassDefinition(name, glyphs) - - self._glyphclasses[group.name.lower()] = glyphclass + glyphs = self._coverage(group.enum.enum, flatten=True) + glyphclass = ast.GlyphClass(glyphs) + classdef = ast.GlyphClassDefinition(name, glyphclass) + self._glyphclasses[group.name.lower()] = classdef def _glyphDefinition(self, glyph): try: @@ -317,10 +368,10 @@ class VoltToFea: assert ltag not in self._features[ftag][stag] self._features[ftag][stag][ltag] = lookups.keys() - def _settingDefinition(self, setting): + def _settingDefinition(self, setting, ignore_unsupported=False): if setting.name.startswith("COMPILER_"): self._settings[setting.name] = setting.value - else: + elif not ignore_unsupported: log.warning(f"Unsupported setting ignored: {setting.name}") def _adjustment(self, adjustment): @@ -358,18 +409,15 @@ class VoltToFea: glyphname = anchordef.glyph_name anchor = self._anchor(anchordef.pos) + if glyphname not in self._anchors: + self._anchors[glyphname] = {} if anchorname.startswith("MARK_"): - name = "_".join(anchorname.split("_")[1:]) - markclass = ast.MarkClass(self._className(name)) - glyph = self._glyphName(glyphname) - markdef = MarkClassDefinition(markclass, anchor, glyph) - self._markclasses[(glyphname, anchorname)] = markdef + anchorname = anchorname[:5] + anchorname[5:].lower() else: - if glyphname not in self._anchors: - self._anchors[glyphname] = {} - if anchorname not in self._anchors[glyphname]: - self._anchors[glyphname][anchorname] = {} - self._anchors[glyphname][anchorname][anchordef.component] = anchor + anchorname = anchorname.lower() + if anchorname not in self._anchors[glyphname]: + self._anchors[glyphname][anchorname] = {} + self._anchors[glyphname][anchorname][anchordef.component] = anchor def _gposLookup(self, lookup, fealookup): statements = fealookup.statements @@ -408,43 +456,66 @@ class VoltToFea: ) elif isinstance(pos, VAst.PositionAttachDefinition): anchors = {} - for marks, classname in pos.coverage_to: - for mark in marks: - # Set actually used mark classes. Basically a hack to get - # around the feature file syntax limitation of making mark - # classes global and not allowing mark positioning to - # specify mark coverage. - for name in mark.glyphSet(): - key = (name, "MARK_" + classname) - self._markclasses[key].used = True - markclass = ast.MarkClass(self._className(classname)) + allmarks = set() + for coverage, anchorname in pos.coverage_to: + # In feature files mark classes are global, but in VOLT they + # are defined per-lookup. If we output mark class definitions + # for all marks that use a given anchor, we might end up with a + # mark used in two different classes in the same lookup, which + # is causes feature file compilation error. + # At the expense of uglier feature code, we make the mark class + # name by appending the current lookup name not the anchor + # name, and output mark class definitions only for marks used + # in this lookup. + classname = self._className(f"{anchorname}.{lookup.name}") + markclass = ast.MarkClass(classname) + + # Anchor names are case-insensitive in VOLT + anchorname = anchorname.lower() + + # We might still end in marks used in two different anchor + # classes, so we filter out already used marks. + marks = set() + for mark in coverage: + marks.update(mark.glyphSet()) + if not marks.isdisjoint(allmarks): + marks.difference_update(allmarks) + if not marks: + continue + allmarks.update(marks) + + for glyphname in marks: + glyph = self._glyphName(glyphname) + anchor = self._anchors[glyphname][f"MARK_{anchorname}"][1] + markdef = ast.MarkClassDefinition(markclass, anchor, glyph) + self._markclasses[(glyphname, classname)] = markdef + for base in pos.coverage: for name in base.glyphSet(): if name not in anchors: anchors[name] = [] - if classname not in anchors[name]: - anchors[name].append(classname) + if (anchorname, classname) not in anchors[name]: + anchors[name].append((anchorname, classname)) + is_ligature = all(n in self._ligatures for n in anchors) + is_mark = all(n in self._marks for n in anchors) for name in anchors: components = 1 - if name in self._ligatures: + if is_ligature: components = self._ligatures[name] - marks = [] - for mark in anchors[name]: - markclass = ast.MarkClass(self._className(mark)) + marks = [[] for _ in range(components)] + for mark, classname in anchors[name]: + markclass = ast.MarkClass(classname) for component in range(1, components + 1): - if len(marks) < component: - marks.append([]) - anchor = None if component in self._anchors[name][mark]: anchor = self._anchors[name][mark][component] - marks[component - 1].append((anchor, markclass)) + marks[component - 1].append((anchor, markclass)) base = self._glyphName(name) - if name in self._marks: + if is_mark: mark = ast.MarkMarkPosStatement(base, marks[0]) - elif name in self._ligatures: + elif is_ligature: mark = ast.MarkLigPosStatement(base, marks) else: mark = ast.MarkBasePosStatement(base, marks[0]) @@ -481,13 +552,9 @@ class VoltToFea: else: raise NotImplementedError(pos) - def _gposContextLookup( - self, lookup, prefix, suffix, ignore, fealookup, targetlookup - ): + def _gposContextLookup(self, lookup, prefix, suffix, ignore, fealookup, chained): statements = fealookup.statements - assert not lookup.reversal - pos = lookup.pos if isinstance(pos, VAst.PositionAdjustPairDefinition): for (idx1, idx2), (pos1, pos2) in pos.adjust_pair.items(): @@ -500,79 +567,181 @@ class VoltToFea: if ignore: statement = ast.IgnorePosStatement([(prefix, glyphs, suffix)]) else: - lookups = (targetlookup, targetlookup) statement = ast.ChainContextPosStatement( - prefix, glyphs, suffix, lookups + prefix, glyphs, suffix, [chained, chained] ) statements.append(statement) elif isinstance(pos, VAst.PositionAdjustSingleDefinition): glyphs = [ast.GlyphClass()] - for a, b in pos.adjust_single: - glyph = self._coverage(a) - glyphs[0].extend(glyph) + for a, _ in pos.adjust_single: + glyphs[0].extend(self._coverage(a, flatten=True)) if ignore: statement = ast.IgnorePosStatement([(prefix, glyphs, suffix)]) else: statement = ast.ChainContextPosStatement( - prefix, glyphs, suffix, [targetlookup] + prefix, glyphs, suffix, [chained] ) statements.append(statement) elif isinstance(pos, VAst.PositionAttachDefinition): glyphs = [ast.GlyphClass()] for coverage, _ in pos.coverage_to: - glyphs[0].extend(self._coverage(coverage)) + glyphs[0].extend(self._coverage(coverage, flatten=True)) if ignore: statement = ast.IgnorePosStatement([(prefix, glyphs, suffix)]) else: statement = ast.ChainContextPosStatement( - prefix, glyphs, suffix, [targetlookup] + prefix, glyphs, suffix, [chained] ) statements.append(statement) else: raise NotImplementedError(pos) - def _gsubLookup(self, lookup, prefix, suffix, ignore, chain, fealookup): + def _gsubLookup(self, lookup, fealookup): statements = fealookup.statements sub = lookup.sub + + # Alternate substitutions are represented by adding multiple + # substitutions for the same glyph, so we need to collect them into one + # to many mapping. + if isinstance(sub, VAst.SubstitutionAlternateDefinition): + alternates = {} + for key, val in sub.mapping.items(): + if not key or not val: + path, line, column = sub.location + log.warning(f"{path}:{line}:{column}: Ignoring empty substitution") + continue + glyphs = self._coverage(key) + replacements = self._coverage(val) + assert len(glyphs) == 1 + for src_glyph, repl_glyph in zip( + glyphs[0].glyphSet(), replacements[0].glyphSet() + ): + alternates.setdefault(str(self._glyphName(src_glyph)), []).append( + str(self._glyphName(repl_glyph)) + ) + + for glyph, replacements in alternates.items(): + statement = ast.AlternateSubstStatement( + [], glyph, [], ast.GlyphClass(replacements) + ) + statements.append(statement) + return + for key, val in sub.mapping.items(): if not key or not val: path, line, column = sub.location log.warning(f"{path}:{line}:{column}: Ignoring empty substitution") continue - statement = None glyphs = self._coverage(key) replacements = self._coverage(val) - if ignore: - chain_context = (prefix, glyphs, suffix) - statement = ast.IgnoreSubstStatement([chain_context]) - elif isinstance(sub, VAst.SubstitutionSingleDefinition): + if isinstance(sub, VAst.SubstitutionSingleDefinition): assert len(glyphs) == 1 assert len(replacements) == 1 - statement = ast.SingleSubstStatement( - glyphs, replacements, prefix, suffix, chain + statements.append( + ast.SingleSubstStatement(glyphs, replacements, [], [], False) ) elif isinstance(sub, VAst.SubstitutionReverseChainingSingleDefinition): - assert len(glyphs) == 1 - assert len(replacements) == 1 - statement = ast.ReverseChainSingleSubstStatement( - prefix, suffix, glyphs, replacements - ) + # This is handled in gsubContextLookup() + pass elif isinstance(sub, VAst.SubstitutionMultipleDefinition): assert len(glyphs) == 1 - statement = ast.MultipleSubstStatement( - prefix, glyphs[0], suffix, replacements, chain + statements.append( + ast.MultipleSubstStatement([], glyphs[0], [], replacements) ) elif isinstance(sub, VAst.SubstitutionLigatureDefinition): assert len(replacements) == 1 statement = ast.LigatureSubstStatement( - prefix, glyphs, suffix, replacements[0], chain + [], glyphs, [], replacements[0], False ) + + # If any of the input glyphs is a group, we need to + # explode the substitution into multiple ligature substitutions + # since feature file syntax does not support classes in + # ligature substitutions. + n = max(len(x.glyphSet()) for x in glyphs) + if n > 1: + # All input should either be groups of the same length or single glyphs + assert all(len(x.glyphSet()) in (n, 1) for x in glyphs) + glyphs = [x.glyphSet() for x in glyphs] + glyphs = [([x[0]] * n if len(x) == 1 else x) for x in glyphs] + + # In this case ligature replacements must be a group of the same length + # as the input groups, or a single glyph. VOLT + # allows the replacement glyphs to be longer and truncates them. + # So well allow that and zip() below will do the truncation + # for us. + replacement = replacements[0].glyphSet() + if len(replacement) == 1: + replacement = [replacement[0]] * n + assert len(replacement) >= n + + # Add the unexploded statement commented out for reference. + statements.append(ast.Comment(f"# {statement}")) + + for zipped in zip(*glyphs, replacement): + zipped = [self._glyphName(x) for x in zipped] + statements.append( + ast.LigatureSubstStatement( + [], zipped[:-1], [], zipped[-1], False + ) + ) + else: + statements.append(statement) else: raise NotImplementedError(sub) - statements.append(statement) + + def _gsubContextLookup(self, lookup, prefix, suffix, ignore, fealookup, chained): + statements = fealookup.statements + + sub = lookup.sub + + if isinstance(sub, VAst.SubstitutionReverseChainingSingleDefinition): + # Reverse substitutions is a special case, it can’t use chained lookups. + for key, val in sub.mapping.items(): + if not key or not val: + path, line, column = sub.location + log.warning(f"{path}:{line}:{column}: Ignoring empty substitution") + continue + glyphs = self._coverage(key) + replacements = self._coverage(val) + statements.append( + ast.ReverseChainSingleSubstStatement( + prefix, suffix, glyphs, replacements + ) + ) + fealookup.chained = [] + return + + if not isinstance( + sub, + ( + VAst.SubstitutionSingleDefinition, + VAst.SubstitutionMultipleDefinition, + VAst.SubstitutionLigatureDefinition, + VAst.SubstitutionAlternateDefinition, + ), + ): + raise NotImplementedError(type(sub)) + + glyphs = [] + for key, val in sub.mapping.items(): + if not key or not val: + path, line, column = sub.location + log.warning(f"{path}:{line}:{column}: Ignoring empty substitution") + continue + glyphs.extend(self._coverage(key, flatten=True)) + + if len(glyphs) > 1: + glyphs = [ast.GlyphClass(glyphs)] + if ignore: + statements.append(ast.IgnoreSubstStatement([(prefix, glyphs, suffix)])) + else: + statements.append( + ast.ChainContextSubstStatement(prefix, glyphs, suffix, [chained]) + ) def _lookupDefinition(self, lookup): mark_attachement = None @@ -598,13 +767,21 @@ class VoltToFea: lookupflags = ast.LookupFlagStatement( flags, mark_attachement, mark_filtering ) + + use_extension = False + if self._settings.get("COMPILER_USEEXTENSIONLOOKUPS"): + use_extension = True + if "\\" in lookup.name: # Merge sub lookups as subtables (lookups named “base\sub”), # makeotf/feaLib will issue a warning and ignore the subtable # statement if it is not a pairpos lookup, though. name = lookup.name.split("\\")[0] if name.lower() not in self._lookups: - fealookup = ast.LookupBlock(self._lookupName(name)) + fealookup = Lookup( + self._lookupName(name), + use_extension=use_extension, + ) if lookupflags is not None: fealookup.statements.append(lookupflags) fealookup.statements.append(ast.Comment("# " + lookup.name)) @@ -614,7 +791,10 @@ class VoltToFea: fealookup.statements.append(ast.Comment("# " + lookup.name)) self._lookups[name.lower()] = fealookup else: - fealookup = ast.LookupBlock(self._lookupName(lookup.name)) + fealookup = Lookup( + self._lookupName(lookup.name), + use_extension=use_extension, + ) if lookupflags is not None: fealookup.statements.append(lookupflags) self._lookups[lookup.name.lower()] = fealookup @@ -623,39 +803,40 @@ class VoltToFea: fealookup.statements.append(ast.Comment("# " + lookup.comments)) contexts = [] - if lookup.context: - for context in lookup.context: - prefix = self._context(context.left) - suffix = self._context(context.right) - ignore = context.ex_or_in == "EXCEPT_CONTEXT" - contexts.append([prefix, suffix, ignore, False]) - # It seems that VOLT will create contextual substitution using - # only the input if there is no other contexts in this lookup. - if ignore and len(lookup.context) == 1: - contexts.append([[], [], False, True]) - else: - contexts.append([[], [], False, False]) - - targetlookup = None - for prefix, suffix, ignore, chain in contexts: + for context in lookup.context: + prefix = self._context(context.left) + suffix = self._context(context.right) + ignore = context.ex_or_in == "EXCEPT_CONTEXT" + contexts.append([prefix, suffix, ignore]) + # It seems that VOLT will create contextual substitution using + # only the input if there is no other contexts in this lookup. + if ignore and len(lookup.context) == 1: + contexts.append([[], [], False]) + + if contexts: + chained = ast.LookupBlock( + self._lookupName(lookup.name + " chained"), + use_extension=use_extension, + ) + fealookup.chained.append(chained) if lookup.sub is not None: - self._gsubLookup(lookup, prefix, suffix, ignore, chain, fealookup) - - if lookup.pos is not None: - if self._settings.get("COMPILER_USEEXTENSIONLOOKUPS"): - fealookup.use_extension = True - if prefix or suffix or chain or ignore: - if not ignore and targetlookup is None: - targetname = self._lookupName(lookup.name + " target") - targetlookup = ast.LookupBlock(targetname) - fealookup.targets = getattr(fealookup, "targets", []) - fealookup.targets.append(targetlookup) - self._gposLookup(lookup, targetlookup) + self._gsubLookup(lookup, chained) + elif lookup.pos is not None: + self._gposLookup(lookup, chained) + for prefix, suffix, ignore in contexts: + if lookup.sub is not None: + self._gsubContextLookup( + lookup, prefix, suffix, ignore, fealookup, chained + ) + elif lookup.pos is not None: self._gposContextLookup( - lookup, prefix, suffix, ignore, fealookup, targetlookup + lookup, prefix, suffix, ignore, fealookup, chained ) - else: - self._gposLookup(lookup, fealookup) + else: + if lookup.sub is not None: + self._gsubLookup(lookup, fealookup) + elif lookup.pos is not None: + self._gposLookup(lookup, fealookup) def main(args=None): diff --git a/contrib/python/fonttools/ya.make b/contrib/python/fonttools/ya.make index 2c3763e30b8..f8dcab48540 100644 --- a/contrib/python/fonttools/ya.make +++ b/contrib/python/fonttools/ya.make @@ -2,7 +2,7 @@ PY3_LIBRARY() -VERSION(4.57.0) +VERSION(4.58.0) LICENSE(MIT) @@ -192,6 +192,7 @@ PY_SRCS( fontTools/ttLib/tables/G_P_K_G_.py fontTools/ttLib/tables/G_P_O_S_.py fontTools/ttLib/tables/G_S_U_B_.py + fontTools/ttLib/tables/G_V_A_R_.py fontTools/ttLib/tables/G__l_a_t.py fontTools/ttLib/tables/G__l_o_c.py fontTools/ttLib/tables/H_V_A_R_.py @@ -289,6 +290,7 @@ PY_SRCS( fontTools/ufoLib/validators.py fontTools/unicode.py fontTools/unicodedata/Blocks.py + fontTools/unicodedata/Mirrored.py fontTools/unicodedata/OTTags.py fontTools/unicodedata/ScriptExtensions.py fontTools/unicodedata/Scripts.py @@ -323,6 +325,7 @@ PY_SRCS( fontTools/varLib/stat.py fontTools/varLib/varStore.py fontTools/voltLib/__init__.py + fontTools/voltLib/__main__.py fontTools/voltLib/ast.py fontTools/voltLib/error.py fontTools/voltLib/lexer.py diff --git a/contrib/python/platformdirs/.dist-info/METADATA b/contrib/python/platformdirs/.dist-info/METADATA index 91c59c9a280..d8668fed1ea 100644 --- a/contrib/python/platformdirs/.dist-info/METADATA +++ b/contrib/python/platformdirs/.dist-info/METADATA @@ -1,6 +1,6 @@ -Metadata-Version: 2.3 +Metadata-Version: 2.4 Name: platformdirs -Version: 4.3.6 +Version: 4.3.8 Summary: A small Python package for determining appropriate platform-specific dirs, e.g. a `user data dir`. Project-URL: Changelog, https://github.com/tox-dev/platformdirs/releases Project-URL: Documentation, https://platformdirs.readthedocs.io @@ -17,7 +17,6 @@ Classifier: License :: OSI Approved :: MIT License Classifier: Operating System :: OS Independent Classifier: Programming Language :: Python Classifier: Programming Language :: Python :: 3 :: Only -Classifier: Programming Language :: Python :: 3.8 Classifier: Programming Language :: Python :: 3.9 Classifier: Programming Language :: Python :: 3.10 Classifier: Programming Language :: Python :: 3.11 @@ -26,20 +25,20 @@ Classifier: Programming Language :: Python :: 3.13 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy Classifier: Topic :: Software Development :: Libraries :: Python Modules -Requires-Python: >=3.8 +Requires-Python: >=3.9 Provides-Extra: docs Requires-Dist: furo>=2024.8.6; extra == 'docs' Requires-Dist: proselint>=0.14; extra == 'docs' -Requires-Dist: sphinx-autodoc-typehints>=2.4; extra == 'docs' -Requires-Dist: sphinx>=8.0.2; extra == 'docs' +Requires-Dist: sphinx-autodoc-typehints>=3; extra == 'docs' +Requires-Dist: sphinx>=8.1.3; extra == 'docs' Provides-Extra: test Requires-Dist: appdirs==1.4.4; extra == 'test' Requires-Dist: covdefaults>=2.3; extra == 'test' -Requires-Dist: pytest-cov>=5; extra == 'test' +Requires-Dist: pytest-cov>=6; extra == 'test' Requires-Dist: pytest-mock>=3.14; extra == 'test' -Requires-Dist: pytest>=8.3.2; extra == 'test' +Requires-Dist: pytest>=8.3.4; extra == 'test' Provides-Extra: type -Requires-Dist: mypy>=1.11.2; extra == 'type' +Requires-Dist: mypy>=1.14.1; extra == 'type' Description-Content-Type: text/x-rst The problem @@ -115,10 +114,14 @@ On macOS: >>> appauthor = "Acme" >>> user_data_dir(appname, appauthor) '/Users/trentm/Library/Application Support/SuperApp' - >>> site_data_dir(appname, appauthor) - '/Library/Application Support/SuperApp' + >>> user_config_dir(appname, appauthor) + '/Users/trentm/Library/Application Support/SuperApp' >>> user_cache_dir(appname, appauthor) '/Users/trentm/Library/Caches/SuperApp' + >>> site_data_dir(appname, appauthor) + '/Library/Application Support/SuperApp' + >>> site_config_dir(appname, appauthor) + '/Library/Application Support/SuperApp' >>> user_log_dir(appname, appauthor) '/Users/trentm/Library/Logs/SuperApp' >>> user_documents_dir() @@ -147,8 +150,14 @@ On Windows: 'C:\\Users\\trentm\\AppData\\Local\\Acme\\SuperApp' >>> user_data_dir(appname, appauthor, roaming=True) 'C:\\Users\\trentm\\AppData\\Roaming\\Acme\\SuperApp' + >>> user_config_dir(appname, appauthor) + 'C:\\Users\\trentm\\AppData\\Local\\Acme\\SuperApp' >>> user_cache_dir(appname, appauthor) 'C:\\Users\\trentm\\AppData\\Local\\Acme\\SuperApp\\Cache' + >>> site_data_dir(appname, appauthor) + 'C:\\ProgramData\\Acme\\SuperApp' + >>> site_config_dir(appname, appauthor) + 'C:\\ProgramData\\Acme\\SuperApp' >>> user_log_dir(appname, appauthor) 'C:\\Users\\trentm\\AppData\\Local\\Acme\\SuperApp\\Logs' >>> user_documents_dir() @@ -175,16 +184,21 @@ On Linux: >>> appauthor = "Acme" >>> user_data_dir(appname, appauthor) '/home/trentm/.local/share/SuperApp' + >>> user_config_dir(appname) + '/home/trentm/.config/SuperApp' + >>> user_cache_dir(appname, appauthor) + '/home/trentm/.cache/SuperApp' >>> site_data_dir(appname, appauthor) '/usr/local/share/SuperApp' >>> site_data_dir(appname, appauthor, multipath=True) '/usr/local/share/SuperApp:/usr/share/SuperApp' - >>> user_cache_dir(appname, appauthor) - '/home/trentm/.cache/SuperApp' + >>> site_config_dir(appname) + '/etc/xdg/SuperApp' + >>> os.environ["XDG_CONFIG_DIRS"] = "/etc:/usr/local/etc" + >>> site_config_dir(appname, multipath=True) + '/etc/SuperApp:/usr/local/etc/SuperApp' >>> user_log_dir(appname, appauthor) '/home/trentm/.local/state/SuperApp/log' - >>> user_config_dir(appname) - '/home/trentm/.config/SuperApp' >>> user_documents_dir() '/home/trentm/Documents' >>> user_downloads_dir() @@ -199,11 +213,6 @@ On Linux: '/home/trentm/Desktop' >>> user_runtime_dir(appname, appauthor) '/run/user/{os.getuid()}/SuperApp' - >>> site_config_dir(appname) - '/etc/xdg/SuperApp' - >>> os.environ["XDG_CONFIG_DIRS"] = "/etc:/usr/local/etc" - >>> site_config_dir(appname, multipath=True) - '/etc/SuperApp:/usr/local/etc/SuperApp' On Android:: @@ -212,12 +221,16 @@ On Android:: >>> appauthor = "Acme" >>> user_data_dir(appname, appauthor) '/data/data/com.myApp/files/SuperApp' + >>> user_config_dir(appname) + '/data/data/com.myApp/shared_prefs/SuperApp' >>> user_cache_dir(appname, appauthor) '/data/data/com.myApp/cache/SuperApp' + >>> site_data_dir(appname, appauthor) + '/data/data/com.myApp/files/SuperApp' + >>> site_config_dir(appname) + '/data/data/com.myApp/shared_prefs/SuperApp' >>> user_log_dir(appname, appauthor) '/data/data/com.myApp/cache/SuperApp/log' - >>> user_config_dir(appname) - '/data/data/com.myApp/shared_prefs/SuperApp' >>> user_documents_dir() '/storage/emulated/0/Documents' >>> user_downloads_dir() @@ -249,8 +262,14 @@ apps also support ``XDG_*`` environment variables. >>> dirs = PlatformDirs("SuperApp", "Acme") >>> dirs.user_data_dir '/Users/trentm/Library/Application Support/SuperApp' + >>> dirs.user_config_dir + '/Users/trentm/Library/Application Support/SuperApp' + >>> dirs.user_cache_dir + '/Users/trentm/Library/Caches/SuperApp' >>> dirs.site_data_dir '/Library/Application Support/SuperApp' + >>> dirs.site_config_dir + '/Library/Application Support/SuperApp' >>> dirs.user_cache_dir '/Users/trentm/Library/Caches/SuperApp' >>> dirs.user_log_dir @@ -281,10 +300,14 @@ dirs:: >>> dirs = PlatformDirs("SuperApp", "Acme", version="1.0") >>> dirs.user_data_dir '/Users/trentm/Library/Application Support/SuperApp/1.0' - >>> dirs.site_data_dir - '/Library/Application Support/SuperApp/1.0' + >>> dirs.user_config_dir + '/Users/trentm/Library/Application Support/SuperApp/1.0' >>> dirs.user_cache_dir '/Users/trentm/Library/Caches/SuperApp/1.0' + >>> dirs.site_data_dir + '/Library/Application Support/SuperApp/1.0' + >>> dirs.site_config_dir + '/Library/Application Support/SuperApp/1.0' >>> dirs.user_log_dir '/Users/trentm/Library/Logs/SuperApp/1.0' >>> dirs.user_documents_dir diff --git a/contrib/python/platformdirs/README.rst b/contrib/python/platformdirs/README.rst index 1562ecb5906..b83425e17c6 100644 --- a/contrib/python/platformdirs/README.rst +++ b/contrib/python/platformdirs/README.rst @@ -71,10 +71,14 @@ On macOS: >>> appauthor = "Acme" >>> user_data_dir(appname, appauthor) '/Users/trentm/Library/Application Support/SuperApp' - >>> site_data_dir(appname, appauthor) - '/Library/Application Support/SuperApp' + >>> user_config_dir(appname, appauthor) + '/Users/trentm/Library/Application Support/SuperApp' >>> user_cache_dir(appname, appauthor) '/Users/trentm/Library/Caches/SuperApp' + >>> site_data_dir(appname, appauthor) + '/Library/Application Support/SuperApp' + >>> site_config_dir(appname, appauthor) + '/Library/Application Support/SuperApp' >>> user_log_dir(appname, appauthor) '/Users/trentm/Library/Logs/SuperApp' >>> user_documents_dir() @@ -103,8 +107,14 @@ On Windows: 'C:\\Users\\trentm\\AppData\\Local\\Acme\\SuperApp' >>> user_data_dir(appname, appauthor, roaming=True) 'C:\\Users\\trentm\\AppData\\Roaming\\Acme\\SuperApp' + >>> user_config_dir(appname, appauthor) + 'C:\\Users\\trentm\\AppData\\Local\\Acme\\SuperApp' >>> user_cache_dir(appname, appauthor) 'C:\\Users\\trentm\\AppData\\Local\\Acme\\SuperApp\\Cache' + >>> site_data_dir(appname, appauthor) + 'C:\\ProgramData\\Acme\\SuperApp' + >>> site_config_dir(appname, appauthor) + 'C:\\ProgramData\\Acme\\SuperApp' >>> user_log_dir(appname, appauthor) 'C:\\Users\\trentm\\AppData\\Local\\Acme\\SuperApp\\Logs' >>> user_documents_dir() @@ -131,16 +141,21 @@ On Linux: >>> appauthor = "Acme" >>> user_data_dir(appname, appauthor) '/home/trentm/.local/share/SuperApp' + >>> user_config_dir(appname) + '/home/trentm/.config/SuperApp' + >>> user_cache_dir(appname, appauthor) + '/home/trentm/.cache/SuperApp' >>> site_data_dir(appname, appauthor) '/usr/local/share/SuperApp' >>> site_data_dir(appname, appauthor, multipath=True) '/usr/local/share/SuperApp:/usr/share/SuperApp' - >>> user_cache_dir(appname, appauthor) - '/home/trentm/.cache/SuperApp' + >>> site_config_dir(appname) + '/etc/xdg/SuperApp' + >>> os.environ["XDG_CONFIG_DIRS"] = "/etc:/usr/local/etc" + >>> site_config_dir(appname, multipath=True) + '/etc/SuperApp:/usr/local/etc/SuperApp' >>> user_log_dir(appname, appauthor) '/home/trentm/.local/state/SuperApp/log' - >>> user_config_dir(appname) - '/home/trentm/.config/SuperApp' >>> user_documents_dir() '/home/trentm/Documents' >>> user_downloads_dir() @@ -155,11 +170,6 @@ On Linux: '/home/trentm/Desktop' >>> user_runtime_dir(appname, appauthor) '/run/user/{os.getuid()}/SuperApp' - >>> site_config_dir(appname) - '/etc/xdg/SuperApp' - >>> os.environ["XDG_CONFIG_DIRS"] = "/etc:/usr/local/etc" - >>> site_config_dir(appname, multipath=True) - '/etc/SuperApp:/usr/local/etc/SuperApp' On Android:: @@ -168,12 +178,16 @@ On Android:: >>> appauthor = "Acme" >>> user_data_dir(appname, appauthor) '/data/data/com.myApp/files/SuperApp' + >>> user_config_dir(appname) + '/data/data/com.myApp/shared_prefs/SuperApp' >>> user_cache_dir(appname, appauthor) '/data/data/com.myApp/cache/SuperApp' + >>> site_data_dir(appname, appauthor) + '/data/data/com.myApp/files/SuperApp' + >>> site_config_dir(appname) + '/data/data/com.myApp/shared_prefs/SuperApp' >>> user_log_dir(appname, appauthor) '/data/data/com.myApp/cache/SuperApp/log' - >>> user_config_dir(appname) - '/data/data/com.myApp/shared_prefs/SuperApp' >>> user_documents_dir() '/storage/emulated/0/Documents' >>> user_downloads_dir() @@ -205,8 +219,14 @@ apps also support ``XDG_*`` environment variables. >>> dirs = PlatformDirs("SuperApp", "Acme") >>> dirs.user_data_dir '/Users/trentm/Library/Application Support/SuperApp' + >>> dirs.user_config_dir + '/Users/trentm/Library/Application Support/SuperApp' + >>> dirs.user_cache_dir + '/Users/trentm/Library/Caches/SuperApp' >>> dirs.site_data_dir '/Library/Application Support/SuperApp' + >>> dirs.site_config_dir + '/Library/Application Support/SuperApp' >>> dirs.user_cache_dir '/Users/trentm/Library/Caches/SuperApp' >>> dirs.user_log_dir @@ -237,10 +257,14 @@ dirs:: >>> dirs = PlatformDirs("SuperApp", "Acme", version="1.0") >>> dirs.user_data_dir '/Users/trentm/Library/Application Support/SuperApp/1.0' - >>> dirs.site_data_dir - '/Library/Application Support/SuperApp/1.0' + >>> dirs.user_config_dir + '/Users/trentm/Library/Application Support/SuperApp/1.0' >>> dirs.user_cache_dir '/Users/trentm/Library/Caches/SuperApp/1.0' + >>> dirs.site_data_dir + '/Library/Application Support/SuperApp/1.0' + >>> dirs.site_config_dir + '/Library/Application Support/SuperApp/1.0' >>> dirs.user_log_dir '/Users/trentm/Library/Logs/SuperApp/1.0' >>> dirs.user_documents_dir diff --git a/contrib/python/platformdirs/platformdirs/__init__.py b/contrib/python/platformdirs/platformdirs/__init__.py index afe8351d203..02daa5914a8 100644 --- a/contrib/python/platformdirs/platformdirs/__init__.py +++ b/contrib/python/platformdirs/platformdirs/__init__.py @@ -52,7 +52,7 @@ AppDirs = PlatformDirs #: Backwards compatibility with appdirs def user_data_dir( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, roaming: bool = False, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -76,7 +76,7 @@ def user_data_dir( def site_data_dir( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, multipath: bool = False, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -100,7 +100,7 @@ def site_data_dir( def user_config_dir( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, roaming: bool = False, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -124,7 +124,7 @@ def user_config_dir( def site_config_dir( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, multipath: bool = False, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -148,7 +148,7 @@ def site_config_dir( def user_cache_dir( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, opinion: bool = True, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -172,7 +172,7 @@ def user_cache_dir( def site_cache_dir( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, opinion: bool = True, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -196,7 +196,7 @@ def site_cache_dir( def user_state_dir( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, roaming: bool = False, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -220,7 +220,7 @@ def user_state_dir( def user_log_dir( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, opinion: bool = True, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -274,7 +274,7 @@ def user_desktop_dir() -> str: def user_runtime_dir( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, opinion: bool = True, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -298,7 +298,7 @@ def user_runtime_dir( def site_runtime_dir( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, opinion: bool = True, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -322,7 +322,7 @@ def site_runtime_dir( def user_data_path( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, roaming: bool = False, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -346,7 +346,7 @@ def user_data_path( def site_data_path( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, multipath: bool = False, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -370,7 +370,7 @@ def site_data_path( def user_config_path( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, roaming: bool = False, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -394,7 +394,7 @@ def user_config_path( def site_config_path( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, multipath: bool = False, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -418,7 +418,7 @@ def site_config_path( def site_cache_path( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, opinion: bool = True, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -442,7 +442,7 @@ def site_cache_path( def user_cache_path( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, opinion: bool = True, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -466,7 +466,7 @@ def user_cache_path( def user_state_path( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, roaming: bool = False, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -490,7 +490,7 @@ def user_state_path( def user_log_path( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, opinion: bool = True, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -544,7 +544,7 @@ def user_desktop_path() -> Path: def user_runtime_path( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, opinion: bool = True, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 @@ -568,7 +568,7 @@ def user_runtime_path( def site_runtime_path( appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, opinion: bool = True, # noqa: FBT001, FBT002 ensure_exists: bool = False, # noqa: FBT001, FBT002 diff --git a/contrib/python/platformdirs/platformdirs/android.py b/contrib/python/platformdirs/platformdirs/android.py index 7004a852422..92efc852d38 100644 --- a/contrib/python/platformdirs/platformdirs/android.py +++ b/contrib/python/platformdirs/platformdirs/android.py @@ -23,7 +23,7 @@ class Android(PlatformDirsABC): @property def user_data_dir(self) -> str: """:return: data directory tied to the user, e.g. ``/data/user/<userid>/<packagename>/files/<AppName>``""" - return self._append_app_name_and_version(cast(str, _android_folder()), "files") + return self._append_app_name_and_version(cast("str", _android_folder()), "files") @property def site_data_dir(self) -> str: @@ -36,7 +36,7 @@ class Android(PlatformDirsABC): :return: config directory tied to the user, e.g. \ ``/data/user/<userid>/<packagename>/shared_prefs/<AppName>`` """ - return self._append_app_name_and_version(cast(str, _android_folder()), "shared_prefs") + return self._append_app_name_and_version(cast("str", _android_folder()), "shared_prefs") @property def site_config_dir(self) -> str: @@ -46,7 +46,7 @@ class Android(PlatformDirsABC): @property def user_cache_dir(self) -> str: """:return: cache directory tied to the user, e.g.,``/data/user/<userid>/<packagename>/cache/<AppName>``""" - return self._append_app_name_and_version(cast(str, _android_folder()), "cache") + return self._append_app_name_and_version(cast("str", _android_folder()), "cache") @property def site_cache_dir(self) -> str: diff --git a/contrib/python/platformdirs/platformdirs/api.py b/contrib/python/platformdirs/platformdirs/api.py index 18d660e4f8c..a352035ec69 100644 --- a/contrib/python/platformdirs/platformdirs/api.py +++ b/contrib/python/platformdirs/platformdirs/api.py @@ -8,7 +8,8 @@ from pathlib import Path from typing import TYPE_CHECKING if TYPE_CHECKING: - from typing import Iterator, Literal + from collections.abc import Iterator + from typing import Literal class PlatformDirsABC(ABC): # noqa: PLR0904 @@ -17,7 +18,7 @@ class PlatformDirsABC(ABC): # noqa: PLR0904 def __init__( # noqa: PLR0913, PLR0917 self, appname: str | None = None, - appauthor: str | None | Literal[False] = None, + appauthor: str | Literal[False] | None = None, version: str | None = None, roaming: bool = False, # noqa: FBT001, FBT002 multipath: bool = False, # noqa: FBT001, FBT002 diff --git a/contrib/python/platformdirs/platformdirs/unix.py b/contrib/python/platformdirs/platformdirs/unix.py index f1942e92ef4..fc75d8d0747 100644 --- a/contrib/python/platformdirs/platformdirs/unix.py +++ b/contrib/python/platformdirs/platformdirs/unix.py @@ -6,10 +6,13 @@ import os import sys from configparser import ConfigParser from pathlib import Path -from typing import Iterator, NoReturn +from typing import TYPE_CHECKING, NoReturn from .api import PlatformDirsABC +if TYPE_CHECKING: + from collections.abc import Iterator + if sys.platform == "win32": def getuid() -> NoReturn: diff --git a/contrib/python/platformdirs/platformdirs/version.py b/contrib/python/platformdirs/platformdirs/version.py index afb49243e3d..611ac615443 100644 --- a/contrib/python/platformdirs/platformdirs/version.py +++ b/contrib/python/platformdirs/platformdirs/version.py @@ -1,8 +1,13 @@ -# file generated by setuptools_scm +# file generated by setuptools-scm # don't change, don't track in version control + +__all__ = ["__version__", "__version_tuple__", "version", "version_tuple"] + TYPE_CHECKING = False if TYPE_CHECKING: - from typing import Tuple, Union + from typing import Tuple + from typing import Union + VERSION_TUPLE = Tuple[Union[int, str], ...] else: VERSION_TUPLE = object @@ -12,5 +17,5 @@ __version__: str __version_tuple__: VERSION_TUPLE version_tuple: VERSION_TUPLE -__version__ = version = '4.3.6' -__version_tuple__ = version_tuple = (4, 3, 6) +__version__ = version = '4.3.8' +__version_tuple__ = version_tuple = (4, 3, 8) diff --git a/contrib/python/platformdirs/ya.make b/contrib/python/platformdirs/ya.make index 66109cf6a8b..1a33df4191e 100644 --- a/contrib/python/platformdirs/ya.make +++ b/contrib/python/platformdirs/ya.make @@ -2,7 +2,7 @@ PY3_LIBRARY() -VERSION(4.3.6) +VERSION(4.3.8) LICENSE(MIT) diff --git a/contrib/python/types-protobuf/.dist-info/METADATA b/contrib/python/types-protobuf/.dist-info/METADATA index d0b3a5d6eb4..0fd53bbd2c4 100644 --- a/contrib/python/types-protobuf/.dist-info/METADATA +++ b/contrib/python/types-protobuf/.dist-info/METADATA @@ -1,27 +1,19 @@ -Metadata-Version: 2.2 +Metadata-Version: 2.4 Name: types-protobuf -Version: 5.29.1.20250208 +Version: 6.30.2.20250506 Summary: Typing stubs for protobuf -Home-page: https://github.com/python/typeshed -License: Apache-2.0 +License-Expression: Apache-2.0 +Project-URL: Homepage, https://github.com/python/typeshed Project-URL: GitHub, https://github.com/python/typeshed Project-URL: Changes, https://github.com/typeshed-internal/stub_uploader/blob/main/data/changelogs/protobuf.md Project-URL: Issue tracker, https://github.com/python/typeshed/issues Project-URL: Chat, https://gitter.im/python/typing -Classifier: License :: OSI Approved :: Apache Software License Classifier: Programming Language :: Python :: 3 Classifier: Typing :: Stubs Only Requires-Python: >=3.9 Description-Content-Type: text/markdown License-File: LICENSE -Dynamic: classifier -Dynamic: description -Dynamic: description-content-type -Dynamic: home-page -Dynamic: license -Dynamic: project-url -Dynamic: requires-python -Dynamic: summary +Dynamic: license-file ## Typing stubs for protobuf @@ -34,9 +26,9 @@ It can be used by type-checking tools like [Pyre](https://pyre-check.org/), PyCharm, etc. to check code that uses `protobuf`. This version of `types-protobuf` aims to provide accurate annotations for -`protobuf~=5.29.1`. +`protobuf~=6.30.2`. -Partially generated using [mypy-protobuf==3.6.0](https://github.com/nipunn1313/mypy-protobuf/tree/v3.6.0) and libprotoc 28.1 on [protobuf v29.1](https://github.com/protocolbuffers/protobuf/releases/tag/v29.1) (python `protobuf==5.29.1`). +Partially generated using [mypy-protobuf==3.6.0](https://github.com/nipunn1313/mypy-protobuf/tree/v3.6.0) and libprotoc 29.0 on [protobuf v30.2](https://github.com/protocolbuffers/protobuf/releases/tag/v30.2) (python `protobuf==6.30.2`). This stub package is marked as [partial](https://peps.python.org/pep-0561/#partial-stub-packages). If you find that annotations are missing, feel free to contribute and help complete them. @@ -51,7 +43,7 @@ directory. This package was tested with mypy 1.15.0, -pyright 1.1.389, +pyright 1.1.400, and pytype 2024.10.11. It was generated from typeshed commit -[`73ebb9dfd7dfce93c5becde4dcdd51d5626853b8`](https://github.com/python/typeshed/commit/73ebb9dfd7dfce93c5becde4dcdd51d5626853b8). +[`4265ee7c72f476e7156949e55784fd82b40e6953`](https://github.com/python/typeshed/commit/4265ee7c72f476e7156949e55784fd82b40e6953). diff --git a/contrib/python/types-protobuf/README.md b/contrib/python/types-protobuf/README.md index 6b33aeb7279..c35bf581ad2 100644 --- a/contrib/python/types-protobuf/README.md +++ b/contrib/python/types-protobuf/README.md @@ -9,9 +9,9 @@ It can be used by type-checking tools like [Pyre](https://pyre-check.org/), PyCharm, etc. to check code that uses `protobuf`. This version of `types-protobuf` aims to provide accurate annotations for -`protobuf~=5.29.1`. +`protobuf~=6.30.2`. -Partially generated using [mypy-protobuf==3.6.0](https://github.com/nipunn1313/mypy-protobuf/tree/v3.6.0) and libprotoc 28.1 on [protobuf v29.1](https://github.com/protocolbuffers/protobuf/releases/tag/v29.1) (python `protobuf==5.29.1`). +Partially generated using [mypy-protobuf==3.6.0](https://github.com/nipunn1313/mypy-protobuf/tree/v3.6.0) and libprotoc 29.0 on [protobuf v30.2](https://github.com/protocolbuffers/protobuf/releases/tag/v30.2) (python `protobuf==6.30.2`). This stub package is marked as [partial](https://peps.python.org/pep-0561/#partial-stub-packages). If you find that annotations are missing, feel free to contribute and help complete them. @@ -26,7 +26,7 @@ directory. This package was tested with mypy 1.15.0, -pyright 1.1.389, +pyright 1.1.400, and pytype 2024.10.11. It was generated from typeshed commit -[`73ebb9dfd7dfce93c5becde4dcdd51d5626853b8`](https://github.com/python/typeshed/commit/73ebb9dfd7dfce93c5becde4dcdd51d5626853b8).
\ No newline at end of file +[`4265ee7c72f476e7156949e55784fd82b40e6953`](https://github.com/python/typeshed/commit/4265ee7c72f476e7156949e55784fd82b40e6953).
\ No newline at end of file diff --git a/contrib/python/types-protobuf/google-stubs/METADATA.toml b/contrib/python/types-protobuf/google-stubs/METADATA.toml index 04c674e5fbc..a7416507152 100644 --- a/contrib/python/types-protobuf/google-stubs/METADATA.toml +++ b/contrib/python/types-protobuf/google-stubs/METADATA.toml @@ -1,7 +1,8 @@ # Using an exact number in the specifier for scripts/sync_protobuf/google_protobuf.py -version = "~=5.29.1" +# When updating, also re-run the script +version = "~=6.30.2" upstream_repository = "https://github.com/protocolbuffers/protobuf" -extra_description = "Partially generated using [mypy-protobuf==3.6.0](https://github.com/nipunn1313/mypy-protobuf/tree/v3.6.0) and libprotoc 28.1 on [protobuf v29.1](https://github.com/protocolbuffers/protobuf/releases/tag/v29.1) (python `protobuf==5.29.1`)." +extra_description = "Partially generated using [mypy-protobuf==3.6.0](https://github.com/nipunn1313/mypy-protobuf/tree/v3.6.0) and libprotoc 29.0 on [protobuf v30.2](https://github.com/protocolbuffers/protobuf/releases/tag/v30.2) (python `protobuf==6.30.2`)." partial_stub = true [tool.stubtest] diff --git a/contrib/python/types-protobuf/google-stubs/_upb/_message.pyi b/contrib/python/types-protobuf/google-stubs/_upb/_message.pyi new file mode 100644 index 00000000000..bb188bcbf4f --- /dev/null +++ b/contrib/python/types-protobuf/google-stubs/_upb/_message.pyi @@ -0,0 +1,310 @@ +from _typeshed import Incomplete +from typing import ClassVar, final + +default_pool: DescriptorPool + +@final +class Arena: ... + +@final +class Descriptor: + containing_type: Incomplete + enum_types: Incomplete + enum_types_by_name: Incomplete + enum_values_by_name: Incomplete + extension_ranges: Incomplete + extensions: Incomplete + extensions_by_name: Incomplete + fields: Incomplete + fields_by_camelcase_name: Incomplete + fields_by_name: Incomplete + fields_by_number: Incomplete + file: Incomplete + full_name: Incomplete + has_options: Incomplete + is_extendable: Incomplete + name: Incomplete + nested_types: Incomplete + nested_types_by_name: Incomplete + oneofs: Incomplete + oneofs_by_name: Incomplete + def __init__(self, *args, **kwargs) -> None: ... # incomplete + def CopyToProto(self, object, /): ... + def EnumValueName(self, *args, **kwargs): ... # incomplete + def GetOptions(self): ... + +@final +class DescriptorPool: + def __init__(self, *args, **kwargs) -> None: ... # incomplete + def Add(self, object, /): ... + def AddSerializedFile(self, object, /): ... + def FindAllExtensions(self, object, /): ... + def FindEnumTypeByName(self, object, /): ... + def FindExtensionByName(self, object, /): ... + def FindExtensionByNumber(self, *args, **kwargs): ... # incomplete + def FindFieldByName(self, object, /): ... + def FindFileByName(self, object, /): ... + def FindFileContainingSymbol(self, object, /): ... + def FindMessageTypeByName(self, object, /): ... + def FindMethodByName(self, object, /): ... + def FindOneofByName(self, object, /): ... + def FindServiceByName(self, object, /): ... + def SetFeatureSetDefaults(self, object, /): ... + +@final +class EnumDescriptor: + containing_type: Incomplete + file: Incomplete + full_name: Incomplete + has_options: Incomplete + is_closed: Incomplete + name: Incomplete + values: Incomplete + values_by_name: Incomplete + values_by_number: Incomplete + def __init__(self, *args, **kwargs) -> None: ... # incomplete + def CopyToProto(self, object, /): ... + def GetOptions(self): ... + +@final +class EnumValueDescriptor: + has_options: Incomplete + index: Incomplete + name: Incomplete + number: Incomplete + type: Incomplete + def __init__(self, *args, **kwargs) -> None: ... # incomplete + def GetOptions(self): ... + +@final +class ExtensionDict: + def __contains__(self, other) -> bool: ... + def __delitem__(self, other) -> None: ... + def __eq__(self, other: object) -> bool: ... + def __ge__(self, other: object) -> bool: ... + def __getitem__(self, index): ... + def __gt__(self, other: object) -> bool: ... + def __iter__(self): ... + def __le__(self, other: object) -> bool: ... + def __len__(self) -> int: ... + def __lt__(self, other: object) -> bool: ... + def __ne__(self, other: object) -> bool: ... + def __setitem__(self, index, object) -> None: ... + +@final +class ExtensionIterator: + def __iter__(self): ... + def __next__(self): ... + +@final +class FieldDescriptor: + CPPTYPE_BOOL: ClassVar[int] = ... + CPPTYPE_BYTES: ClassVar[int] = ... + CPPTYPE_DOUBLE: ClassVar[int] = ... + CPPTYPE_ENUM: ClassVar[int] = ... + CPPTYPE_FLOAT: ClassVar[int] = ... + CPPTYPE_INT32: ClassVar[int] = ... + CPPTYPE_INT64: ClassVar[int] = ... + CPPTYPE_MESSAGE: ClassVar[int] = ... + CPPTYPE_STRING: ClassVar[int] = ... + CPPTYPE_UINT32: ClassVar[int] = ... + CPPTYPE_UINT64: ClassVar[int] = ... + LABEL_OPTIONAL: ClassVar[int] = ... + LABEL_REPEATED: ClassVar[int] = ... + LABEL_REQUIRED: ClassVar[int] = ... + TYPE_BOOL: ClassVar[int] = ... + TYPE_BYTES: ClassVar[int] = ... + TYPE_DOUBLE: ClassVar[int] = ... + TYPE_ENUM: ClassVar[int] = ... + TYPE_FIXED32: ClassVar[int] = ... + TYPE_FIXED64: ClassVar[int] = ... + TYPE_FLOAT: ClassVar[int] = ... + TYPE_GROUP: ClassVar[int] = ... + TYPE_INT32: ClassVar[int] = ... + TYPE_INT64: ClassVar[int] = ... + TYPE_MESSAGE: ClassVar[int] = ... + TYPE_SFIXED32: ClassVar[int] = ... + TYPE_SFIXED64: ClassVar[int] = ... + TYPE_SINT32: ClassVar[int] = ... + TYPE_SINT64: ClassVar[int] = ... + TYPE_STRING: ClassVar[int] = ... + TYPE_UINT32: ClassVar[int] = ... + TYPE_UINT64: ClassVar[int] = ... + camelcase_name: Incomplete + containing_oneof: Incomplete + containing_type: Incomplete + cpp_type: Incomplete + default_value: Incomplete + enum_type: Incomplete + extension_scope: Incomplete + file: Incomplete + full_name: Incomplete + has_default_value: Incomplete + has_options: Incomplete + has_presence: Incomplete + index: Incomplete + is_extension: Incomplete + is_packed: Incomplete + json_name: Incomplete + label: Incomplete + message_type: Incomplete + name: Incomplete + number: Incomplete + type: Incomplete + def __init__(self, *args, **kwargs) -> None: ... # incomplete + def GetOptions(self): ... + +@final +class FileDescriptor: + dependencies: Incomplete + enum_types_by_name: Incomplete + extensions_by_name: Incomplete + has_options: Incomplete + message_types_by_name: Incomplete + name: Incomplete + package: Incomplete + pool: Incomplete + public_dependencies: Incomplete + serialized_pb: Incomplete + services_by_name: Incomplete + def __init__(self, *args, **kwargs) -> None: ... # incomplete + def CopyToProto(self, object, /): ... + def GetOptions(self): ... + +@final +class MapIterator: + def __iter__(self): ... + def __next__(self): ... + +@final +class Message: + Extensions: Incomplete + def __init__(self, *args, **kwargs) -> None: ... # incomplete # incomplete + def ByteSize(self): ... + def Clear(self): ... + def ClearExtension(self, object, /): ... + def ClearField(self, object, /): ... + def CopyFrom(self, object, /): ... + def DiscardUnknownFields(self): ... + def FindInitializationErrors(self): ... + @classmethod + def FromString(cls, object, /): ... + def HasExtension(self, object, /): ... + def HasField(self, object, /): ... + def IsInitialized(self, *args, **kwargs): ... # incomplete + def ListFields(self): ... + def MergeFrom(self, object, /): ... + def MergeFromString(self, object, /): ... + def ParseFromString(self, object, /): ... + def SerializePartialToString(self, *args, **kwargs): ... # incomplete + def SerializeToString(self, *args, **kwargs): ... # incomplete + def SetInParent(self): ... + def UnknownFields(self): ... + def WhichOneof(self, object, /): ... + def __contains__(self, other) -> bool: ... + def __deepcopy__(self, memo=None): ... + def __delattr__(self, name): ... + def __eq__(self, other: object) -> bool: ... + def __ge__(self, other: object) -> bool: ... + def __gt__(self, other: object) -> bool: ... + def __le__(self, other: object) -> bool: ... + def __lt__(self, other: object) -> bool: ... + def __ne__(self, other: object) -> bool: ... + def __setattr__(self, name, value): ... + +@final +class MessageMeta(type): ... + +@final +class MethodDescriptor: + client_streaming: Incomplete + containing_service: Incomplete + full_name: Incomplete + index: Incomplete + input_type: Incomplete + name: Incomplete + output_type: Incomplete + server_streaming: Incomplete + def __init__(self, *args, **kwargs) -> None: ... # incomplete + def CopyToProto(self, object, /): ... + def GetOptions(self): ... + +@final +class OneofDescriptor: + containing_type: Incomplete + fields: Incomplete + full_name: Incomplete + has_options: Incomplete + index: Incomplete + name: Incomplete + def __init__(self, *args, **kwargs) -> None: ... # incomplete + def GetOptions(self): ... + +@final +class RepeatedCompositeContainer: + def __init__(self, *args, **kwargs) -> None: ... # incomplete + def MergeFrom(self, object, /): ... + def add(self, *args, **kwargs): ... # incomplete + def append(self, object, /): ... + def extend(self, object, /): ... + def insert(self, *args, **kwargs): ... # incomplete + def pop(self, *args, **kwargs): ... # incomplete + def remove(self, object, /): ... + def reverse(self): ... + def sort(self, *args, **kwargs): ... # incomplete + def __deepcopy__(self, memo=None): ... + def __delitem__(self, other) -> None: ... + def __eq__(self, other: object) -> bool: ... + def __ge__(self, other: object) -> bool: ... + def __getitem__(self, index): ... + def __gt__(self, other: object) -> bool: ... + def __le__(self, other: object) -> bool: ... + def __len__(self) -> int: ... + def __lt__(self, other: object) -> bool: ... + def __ne__(self, other: object) -> bool: ... + def __setitem__(self, index, object) -> None: ... + +@final +class RepeatedScalarContainer: + def __init__(self, *args, **kwargs) -> None: ... # incomplete + def MergeFrom(self, object, /): ... + def append(self, object, /): ... + def extend(self, object, /): ... + def insert(self, *args, **kwargs): ... # incomplete + def pop(self, *args, **kwargs): ... # incomplete + def remove(self, object, /): ... + def reverse(self): ... + def sort(self, *args, **kwargs): ... # incomplete + def __deepcopy__(self, memo=None): ... + def __delitem__(self, other) -> None: ... + def __eq__(self, other: object) -> bool: ... + def __ge__(self, other: object) -> bool: ... + def __getitem__(self, index): ... + def __gt__(self, other: object) -> bool: ... + def __le__(self, other: object) -> bool: ... + def __len__(self) -> int: ... + def __lt__(self, other: object) -> bool: ... + def __ne__(self, other: object) -> bool: ... + def __reduce__(self): ... + def __setitem__(self, index, object) -> None: ... + +@final +class ServiceDescriptor: + file: Incomplete + full_name: Incomplete + index: Incomplete + methods: Incomplete + methods_by_name: Incomplete + name: Incomplete + def __init__(self, *args, **kwargs) -> None: ... # incomplete + def CopyToProto(self, object, /): ... + def FindMethodByName(self, object, /): ... + def GetOptions(self): ... + +@final +class UnknownFieldSet: + def __init__(self, *args, **kwargs) -> None: ... # incomplete + def __getitem__(self, index): ... + def __len__(self) -> int: ... + +def SetAllowOversizeProtos(object, /): ... # incomplete diff --git a/contrib/python/types-protobuf/google-stubs/protobuf/__init__.pyi b/contrib/python/types-protobuf/google-stubs/protobuf/__init__.pyi index bda5b5a7f4c..c5dd9546606 100644 --- a/contrib/python/types-protobuf/google-stubs/protobuf/__init__.pyi +++ b/contrib/python/types-protobuf/google-stubs/protobuf/__init__.pyi @@ -1 +1,3 @@ -__version__: str +from typing import Final + +__version__: Final[str] diff --git a/contrib/python/types-protobuf/google-stubs/protobuf/descriptor.pyi b/contrib/python/types-protobuf/google-stubs/protobuf/descriptor.pyi index 291ca0c7296..45891fda76c 100644 --- a/contrib/python/types-protobuf/google-stubs/protobuf/descriptor.pyi +++ b/contrib/python/types-protobuf/google-stubs/protobuf/descriptor.pyi @@ -156,6 +156,12 @@ class FieldDescriptor(DescriptorBase): cpp_type: Any @property def label(self): ... + @property + def camelcase_name(self) -> str: ... + @property + def has_presence(self) -> bool: ... + @property + def is_packed(self) -> bool: ... has_default_value: Any default_value: Any containing_type: Any diff --git a/contrib/python/types-protobuf/google-stubs/protobuf/descriptor_database.pyi b/contrib/python/types-protobuf/google-stubs/protobuf/descriptor_database.pyi new file mode 100644 index 00000000000..6568fc2159c --- /dev/null +++ b/contrib/python/types-protobuf/google-stubs/protobuf/descriptor_database.pyi @@ -0,0 +1,16 @@ +from typing import Final + +from google.protobuf.descriptor_pb2 import FileDescriptorProto + +__author__: Final[str] + +class Error(Exception): ... +class DescriptorDatabaseConflictingDefinitionError(Error): ... + +class DescriptorDatabase: + def __init__(self) -> None: ... + def Add(self, file_desc_proto) -> None: ... + def FindFileByName(self, name): ... + def FindFileContainingSymbol(self, symbol): ... + def FindFileContainingExtension(self, extendee_name, extension_number) -> FileDescriptorProto | None: ... + def FindAllExtensionNumbers(self, extendee_name) -> list[int]: ... diff --git a/contrib/python/types-protobuf/google-stubs/protobuf/internal/containers.pyi b/contrib/python/types-protobuf/google-stubs/protobuf/internal/containers.pyi index aaa97043921..75261371607 100644 --- a/contrib/python/types-protobuf/google-stubs/protobuf/internal/containers.pyi +++ b/contrib/python/types-protobuf/google-stubs/protobuf/internal/containers.pyi @@ -1,5 +1,6 @@ from collections.abc import Callable, Iterable, Iterator, MutableMapping, Sequence from typing import Any, Protocol, SupportsIndex, TypeVar, overload +from typing_extensions import Self from google.protobuf.descriptor import Descriptor from google.protobuf.internal.message_listener import MessageListener @@ -10,7 +11,6 @@ _T = TypeVar("_T") _K = TypeVar("_K", bound=bool | int | str) _ScalarV = TypeVar("_ScalarV", bound=bool | int | float | str | bytes) _MessageV = TypeVar("_MessageV", bound=Message) -_M = TypeVar("_M") class _ValueChecker(Protocol[_T]): def CheckValue(self, proposed_value: _T) -> _T: ... @@ -33,7 +33,7 @@ class RepeatedScalarFieldContainer(BaseContainer[_ScalarV]): def append(self, value: _ScalarV) -> None: ... def insert(self, key: int, value: _ScalarV) -> None: ... def extend(self, elem_seq: Iterable[_ScalarV] | None) -> None: ... - def MergeFrom(self: _M, other: _M | Iterable[_ScalarV]) -> None: ... + def MergeFrom(self, other: Self | Iterable[_ScalarV]) -> None: ... def remove(self, elem: _ScalarV) -> None: ... def pop(self, key: int = -1) -> _ScalarV: ... @overload @@ -49,7 +49,7 @@ class RepeatedCompositeFieldContainer(BaseContainer[_MessageV]): def append(self, value: _MessageV) -> None: ... def insert(self, key: int, value: _MessageV) -> None: ... def extend(self, elem_seq: Iterable[_MessageV]) -> None: ... - def MergeFrom(self: _M, other: _M | Iterable[_MessageV]) -> None: ... + def MergeFrom(self, other: Self | Iterable[_MessageV]) -> None: ... def remove(self, elem: _MessageV) -> None: ... def pop(self, key: int = -1) -> _MessageV: ... def __delitem__(self, key: int | slice) -> None: ... @@ -73,7 +73,8 @@ class ScalarMap(MutableMapping[_K, _ScalarV]): def get(self, key: _K, default: None = None) -> _ScalarV | None: ... @overload def get(self, key: _K, default: _ScalarV | _T) -> _ScalarV | _T: ... - def MergeFrom(self: _M, other: _M): ... + def setdefault(self, key: _K, value: _ScalarV | None = None) -> _ScalarV: ... + def MergeFrom(self, other: Self): ... def InvalidateIterators(self) -> None: ... def GetEntryClass(self) -> GeneratedProtocolMessageType: ... @@ -96,6 +97,7 @@ class MessageMap(MutableMapping[_K, _MessageV]): @overload def get(self, key: _K, default: _MessageV | _T) -> _MessageV | _T: ... def get_or_create(self, key: _K) -> _MessageV: ... - def MergeFrom(self: _M, other: _M): ... + def setdefault(self, key: _K, value: _MessageV | None = None) -> _MessageV: ... + def MergeFrom(self, other: Self): ... def InvalidateIterators(self) -> None: ... def GetEntryClass(self) -> GeneratedProtocolMessageType: ... diff --git a/contrib/python/types-protobuf/google-stubs/protobuf/internal/decoder.pyi b/contrib/python/types-protobuf/google-stubs/protobuf/internal/decoder.pyi index ce74e9318c6..94ed7f61536 100644 --- a/contrib/python/types-protobuf/google-stubs/protobuf/internal/decoder.pyi +++ b/contrib/python/types-protobuf/google-stubs/protobuf/internal/decoder.pyi @@ -59,5 +59,3 @@ MESSAGE_SET_ITEM_TAG: bytes def MessageSetItemDecoder(descriptor: Descriptor) -> _Decoder: ... def MapDecoder(field_descriptor, new_default, is_message_map) -> _Decoder: ... - -SkipField: Any diff --git a/contrib/python/types-protobuf/google-stubs/protobuf/message.pyi b/contrib/python/types-protobuf/google-stubs/protobuf/message.pyi index 819ad7aad5d..ea1d636ee26 100644 --- a/contrib/python/types-protobuf/google-stubs/protobuf/message.pyi +++ b/contrib/python/types-protobuf/google-stubs/protobuf/message.pyi @@ -1,7 +1,9 @@ from collections.abc import Sequence -from typing import Any, TypeVar +from typing import Any from typing_extensions import Self +from google._upb._message import Descriptor as _upb_Descriptor + from .descriptor import Descriptor, FieldDescriptor from .internal.extension_dict import _ExtensionDict, _ExtensionFieldDescriptor @@ -9,10 +11,8 @@ class Error(Exception): ... class DecodeError(Error): ... class EncodeError(Error): ... -_M = TypeVar("_M", bound=Message) # message type (of self) - class Message: - DESCRIPTOR: Descriptor + DESCRIPTOR: Descriptor | _upb_Descriptor def __deepcopy__(self, memo: Any = None) -> Self: ... def __eq__(self, other_msg): ... def __ne__(self, other_msg): ... @@ -26,12 +26,11 @@ class Message: def SerializeToString(self, *, deterministic: bool = ...) -> bytes: ... def SerializePartialToString(self, *, deterministic: bool = ...) -> bytes: ... def ListFields(self) -> Sequence[tuple[FieldDescriptor, Any]]: ... - # The TypeVar must be bound to `Message` or we get mypy errors, so we cannot use `Self` for `HasExtension` & `ClearExtension` - def HasExtension(self: _M, field_descriptor: _ExtensionFieldDescriptor[_M, Any]) -> bool: ... - def ClearExtension(self: _M, field_descriptor: _ExtensionFieldDescriptor[_M, Any]) -> None: ... + def HasExtension(self, field_descriptor: _ExtensionFieldDescriptor[Self, Any]) -> bool: ... + def ClearExtension(self, field_descriptor: _ExtensionFieldDescriptor[Self, Any]) -> None: ... # The TypeVar must be bound to `Message` or we get mypy errors, so we cannot use `Self` for `Extensions` @property - def Extensions(self: _M) -> _ExtensionDict[_M]: ... + def Extensions(self) -> _ExtensionDict[Self]: ... def ByteSize(self) -> int: ... @classmethod def FromString(cls, s: bytes) -> Self: ... diff --git a/contrib/python/types-protobuf/google-stubs/protobuf/message_factory.pyi b/contrib/python/types-protobuf/google-stubs/protobuf/message_factory.pyi index 518e1251955..6422284aaad 100644 --- a/contrib/python/types-protobuf/google-stubs/protobuf/message_factory.pyi +++ b/contrib/python/types-protobuf/google-stubs/protobuf/message_factory.pyi @@ -9,8 +9,6 @@ from google.protobuf.message import Message class MessageFactory: pool: Any def __init__(self, pool: DescriptorPool | None = None) -> None: ... - def GetPrototype(self, descriptor: Descriptor) -> type[Message]: ... - def GetMessages(self, files: Iterable[str]) -> dict[str, type[Message]]: ... def GetMessageClass(descriptor: Descriptor) -> type[Message]: ... def GetMessageClassesForFiles(files: Iterable[str], pool: DescriptorPool) -> dict[str, type[Message]]: ... diff --git a/contrib/python/types-protobuf/google-stubs/protobuf/reflection.pyi b/contrib/python/types-protobuf/google-stubs/protobuf/reflection.pyi index 5f7822363b1..2836b3fcf11 100644 --- a/contrib/python/types-protobuf/google-stubs/protobuf/reflection.pyi +++ b/contrib/python/types-protobuf/google-stubs/protobuf/reflection.pyi @@ -1,5 +1,2 @@ class GeneratedProtocolMessageType(type): def __new__(cls, name, bases, dictionary): ... - -def ParseMessage(descriptor, byte_str): ... -def MakeClass(descriptor): ... diff --git a/contrib/python/types-protobuf/google-stubs/protobuf/service.pyi b/contrib/python/types-protobuf/google-stubs/protobuf/service.pyi deleted file mode 100644 index 1123b6134dd..00000000000 --- a/contrib/python/types-protobuf/google-stubs/protobuf/service.pyi +++ /dev/null @@ -1,39 +0,0 @@ -from collections.abc import Callable -from concurrent.futures import Future - -from google.protobuf.descriptor import MethodDescriptor, ServiceDescriptor -from google.protobuf.message import Message - -class RpcException(Exception): ... - -class Service: - @staticmethod - def GetDescriptor() -> ServiceDescriptor: ... - def CallMethod( - self, - method_descriptor: MethodDescriptor, - rpc_controller: RpcController, - request: Message, - done: Callable[[Message], None] | None, - ) -> Future[Message] | None: ... - def GetRequestClass(self, method_descriptor: MethodDescriptor) -> type[Message]: ... - def GetResponseClass(self, method_descriptor: MethodDescriptor) -> type[Message]: ... - -class RpcController: - def Reset(self) -> None: ... - def Failed(self) -> bool: ... - def ErrorText(self) -> str | None: ... - def StartCancel(self) -> None: ... - def SetFailed(self, reason: str) -> None: ... - def IsCanceled(self) -> bool: ... - def NotifyOnCancel(self, callback: Callable[[], None]) -> None: ... - -class RpcChannel: - def CallMethod( - self, - method_descriptor: MethodDescriptor, - rpc_controller: RpcController, - request: Message, - response_class: type[Message], - done: Callable[[Message], None] | None, - ) -> Future[Message] | None: ... diff --git a/contrib/python/types-protobuf/ya.make b/contrib/python/types-protobuf/ya.make index 14901831773..e93e7b16e84 100644 --- a/contrib/python/types-protobuf/ya.make +++ b/contrib/python/types-protobuf/ya.make @@ -2,7 +2,7 @@ PY3_LIBRARY() -VERSION(5.29.1.20250208) +VERSION(6.30.2.20250506) LICENSE(Apache-2.0) @@ -10,9 +10,11 @@ NO_LINT() PY_SRCS( TOP_LEVEL + google-stubs/_upb/_message.pyi google-stubs/protobuf/__init__.pyi google-stubs/protobuf/compiler/__init__.pyi google-stubs/protobuf/descriptor.pyi + google-stubs/protobuf/descriptor_database.pyi google-stubs/protobuf/descriptor_pool.pyi google-stubs/protobuf/internal/__init__.pyi google-stubs/protobuf/internal/api_implementation.pyi @@ -32,7 +34,6 @@ PY_SRCS( google-stubs/protobuf/message_factory.pyi google-stubs/protobuf/reflection.pyi google-stubs/protobuf/runtime_version.pyi - google-stubs/protobuf/service.pyi google-stubs/protobuf/symbol_database.pyi google-stubs/protobuf/text_format.pyi google-stubs/protobuf/util/__init__.pyi |