Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for rendering CJK glyphs top-to-bottom #3402

Closed
wants to merge 26 commits into from
Closed

Add support for rendering CJK glyphs top-to-bottom #3402

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
1c950ee
cjk specific breaking
Aug 11, 2016
cefde62
break evenly for now
Aug 12, 2016
0008ca4
remember all the fixes
Aug 15, 2016
b07e503
wrap several times on super long
Aug 15, 2016
b63fa24
no unused vars
Aug 15, 2016
fe6bbb8
update expected linebreaking
Aug 15, 2016
006dfa8
cjk specific breaking
Aug 11, 2016
46f1d71
break evenly for now
Aug 12, 2016
957edbd
remember all the fixes
Aug 15, 2016
2b73762
started new vertical label implementation and debugging
Sep 7, 2016
6c8135c
more work on vertical labels
Sep 17, 2016
a7dd764
refactor vertical labels to be part of same symbol instances as horiz…
Oct 6, 2016
8012238
updated checks for dropping vertical label glyphs
Oct 17, 2016
1af4500
Remove dead code, commented code, & whitespace changes
Oct 18, 2016
d5f03c5
Restore commented out code.
Oct 19, 2016
c1e0dac
Organize breakable array
Oct 19, 2016
bbb22c3
Misc refactoring of SymbolBucket
Oct 19, 2016
598985a
Fix linter errors
Oct 19, 2016
af45d54
Refactor shaping module for better clarity
Oct 19, 2016
24ac760
Rename "placeText" to "shapeText"
Oct 19, 2016
defb6ef
Rename "quads" to "placeShapedText" / "placeShapedIcons"
Oct 19, 2016
2711569
Build debugging rig
Oct 20, 2016
c82d22b
More better debugging page
Oct 20, 2016
9953d5c
Remove line breaking code
Oct 20, 2016
2f3f5ee
Rename SymbolQuadsArray to PlacedSymbolArray
Oct 20, 2016
b3b65d2
Got placement roughly working
Oct 20, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Rename "placeText" to "shapeText"
  • Loading branch information
Lucas Wojciechowski committed Oct 19, 2016
commit 24ac76040ed8eaa033818831c2031f97c7fc98d3
54 changes: 27 additions & 27 deletions js/data/bucket/symbol_bucket.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,8 @@ var Anchor = require('../../symbol/anchor');
var getAnchors = require('../../symbol/get_anchors');
var resolveTokens = require('../../util/token');
var Quads = require('../../symbol/quads');
var placeText = require('../../symbol/place_text');
var placeIcon = require('../../symbol/place_icon');
var shapeText = require('../../symbol/shape_text');
var shapeIcon = require('../../symbol/shape_icon');
var resolveText = require('../../symbol/resolve_text');
var mergeLines = require('../../symbol/mergelines');
var clipLine = require('../../symbol/clip_line');
Expand Down Expand Up @@ -217,13 +217,13 @@ SymbolBucket.prototype.populateArrays = function(collisionTile, stacks, icons) {
textFeatures = merged.textFeatures;
}

var placedText, placedTextVertical, placedIcon;
var shapedText, shapedTextVertical, shapedIcon;

for (var k = 0; k < features.length; k++) {
if (!geometries[k]) continue;

if (textFeatures[k]) {
placedText = placeText(
shapedText = shapeText(
textFeatures[k],
stacks[fontstack],
maxWidth,
Expand All @@ -238,7 +238,7 @@ SymbolBucket.prototype.populateArrays = function(collisionTile, stacks, icons) {
);

if (layout['text-rotation-alignment'] === 'map' && layout['symbol-placement'] === 'line') {
Copy link
Contributor

@1ec5 1ec5 Oct 18, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid verticalizing non-CJK* text, match textFeatures[k] against this regular expression, courtesy of Wiktionary:

&& !/[^ᄀ-ᇿ가-힣ㄱ-ㆎ一-鿌㐀-䶵　-〿𠀀-𬺯！-￮ぁ-ゟ゠-ヿㇰ-ㇿꀀ-꓆᠀-ᢪ]/.exec(textFeatures[k])

* For the purpose of this PR, “CJK” is Hangul, Hanzi, Hiragana, Katakana, Mongolian, and Yi scripts. However, note that Hangul and Mongolian words are delimited by spaces and thus should retain the Latin-style line breaking algorithm.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(You’ll need to make the regular expression a little more lenient to allow numerals and punctuation.)

Copy link
Contributor

@1ec5 1ec5 Oct 18, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JavaScript can’t handle characters above U+FFFF in character classes – like emoji! 😛 – so the regular expression will need to be a tad more complicated to detect Hanzi from 𠀀 onwards. Specifically, we’ll need to capture anything from \uD840\uDC00 to \uD873\uDEAF, inclusive. (Here’s a very handy tool for calculating surrogate pairs.) My weary Tuesday-evening eyes aren’t helping me come with the correct regex.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@1ec5 thanks, will look at this tomorrow.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@1ec5 Can we use plain ol' inequalities rather than a regex?

Copy link
Contributor

@1ec5 1ec5 Oct 19, 2016 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, whichever is easier to maintain and more performant. It probably doesn’t make a big difference either way.

Character classes can contain \uxxxx character references instead of Unicode literals, if that’s your concern. The surrogate pair issue remains because of JavaScript’s string encoding.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that since we switched to Buble, we can now use u RegExp flag that allows using any unicode characters in regexps (they get transpiled to \u....).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But Babel doesn’t change the fact that JavaScript strings (and therefore regular expressions) must represent characters above U+FFFF as surrogate pairs.

placedTextVertical = placeText(
shapedTextVertical = shapeText(
textFeatures[k],
stacks[fontstack],
maxWidth,
Expand All @@ -257,7 +257,7 @@ SymbolBucket.prototype.populateArrays = function(collisionTile, stacks, icons) {
if (layout['icon-image']) {
var iconName = resolveTokens(features[k].properties, layout['icon-image']);
var image = icons[iconName];
placedIcon = placeIcon(image, layout);
shapedIcon = shapeIcon(image, layout);

if (image) {
if (this.sdfIcons === undefined) {
Expand All @@ -272,11 +272,11 @@ SymbolBucket.prototype.populateArrays = function(collisionTile, stacks, icons) {
}
}
} else {
placedIcon = null;
shapedIcon = null;
}

if (placedText || placedIcon) {
this.addFeature(geometries[k], placedText, placedTextVertical, placedIcon, features[k]);
if (shapedText || shapedIcon) {
this.addFeature(geometries[k], shapedText, shapedTextVertical, shapedIcon, features[k]);
}
}

Expand All @@ -286,7 +286,7 @@ SymbolBucket.prototype.populateArrays = function(collisionTile, stacks, icons) {
this.trimArrays();
};

SymbolBucket.prototype.addFeature = function(lines, placedText, placedTextVertical, placedIcon, feature) {
SymbolBucket.prototype.addFeature = function(lines, shapedText, shapedTextVertical, shapedIcon, feature) {
var layout = this.layer.layout;

var glyphSize = 24;
Expand Down Expand Up @@ -329,9 +329,9 @@ SymbolBucket.prototype.addFeature = function(lines, placedText, placedTextVertic
line,
symbolMinDistance,
textMaxAngle,
placedText,
placedTextVertical,
placedIcon,
shapedText,
shapedTextVertical,
shapedIcon,
glyphSize,
textMaxBoxScale,
this.overscaling,
Expand All @@ -351,8 +351,8 @@ SymbolBucket.prototype.addFeature = function(lines, placedText, placedTextVertic

// this check is commented out because it behaves strangely with vertical labels
// this check should be uncommented before merging this into master.
// if (placedText && isLine) {
// if (this.anchorIsTooClose(placedText.text, textRepeatDistance, anchor)) {
// if (shapedText && isLine) {
// if (this.anchorIsTooClose(shapedText.text, textRepeatDistance, anchor)) {
// continue;
// }
// }
Expand All @@ -375,9 +375,9 @@ SymbolBucket.prototype.addFeature = function(lines, placedText, placedTextVertic
this.addSymbolInstance(
anchor,
line,
placedText,
placedTextVertical,
placedIcon,
shapedText,
shapedTextVertical,
shapedIcon,
this.layer,
addToBuffers,
this.symbolInstancesArray.length,
Expand Down Expand Up @@ -647,17 +647,17 @@ SymbolBucket.prototype.addToDebugBuffers = function(collisionTile) {
}
};

SymbolBucket.prototype.addSymbolInstance = function(anchor, line, placedText, placedTextVertical, placedIcon, layer, addToBuffers, index, collisionBoxArray, featureIndex, sourceLayerIndex, bucketIndex,
SymbolBucket.prototype.addSymbolInstance = function(anchor, line, shapedText, shapedTextVertical, shapedIcon, layer, addToBuffers, index, collisionBoxArray, featureIndex, sourceLayerIndex, bucketIndex,
textBoxScale, textPadding, textAlongLine,
iconBoxScale, iconPadding, iconAlongLine, globalProperties, featureProperties) {

var glyphQuadStartIndex, glyphQuadEndIndex, iconQuadStartIndex, iconQuadEndIndex, textCollisionFeature, iconCollisionFeature, glyphQuads, glyphQuadsVertical, iconQuads;
if (placedText) {
glyphQuads = addToBuffers ? getGlyphQuads(anchor, placedText, textBoxScale, line, layer, textAlongLine, false) : [];
if (placedTextVertical) {
glyphQuadsVertical = addToBuffers ? getGlyphQuads(anchor, placedTextVertical, textBoxScale, line, layer, textAlongLine, true) : [];
if (shapedText) {
glyphQuads = addToBuffers ? getGlyphQuads(anchor, shapedText, textBoxScale, line, layer, textAlongLine, false) : [];
if (shapedTextVertical) {
glyphQuadsVertical = addToBuffers ? getGlyphQuads(anchor, shapedTextVertical, textBoxScale, line, layer, textAlongLine, true) : [];
}
textCollisionFeature = new CollisionFeature(collisionBoxArray, line, anchor, featureIndex, sourceLayerIndex, bucketIndex, placedText, placedTextVertical, textBoxScale, textPadding, textAlongLine, false);
textCollisionFeature = new CollisionFeature(collisionBoxArray, line, anchor, featureIndex, sourceLayerIndex, bucketIndex, shapedText, shapedTextVertical, textBoxScale, textPadding, textAlongLine, false);
}

glyphQuadStartIndex = this.symbolQuadsArray.length;
Expand All @@ -676,9 +676,9 @@ SymbolBucket.prototype.addSymbolInstance = function(anchor, line, placedText, pl
var textBoxStartIndex = textCollisionFeature ? textCollisionFeature.boxStartIndex : this.collisionBoxArray.length;
var textBoxEndIndex = textCollisionFeature ? textCollisionFeature.boxEndIndex : this.collisionBoxArray.length;

if (placedIcon) {
iconQuads = addToBuffers ? getIconQuads(anchor, placedIcon, iconBoxScale, line, layer, iconAlongLine, placedText, globalProperties, featureProperties) : [];
iconCollisionFeature = new CollisionFeature(collisionBoxArray, line, anchor, featureIndex, sourceLayerIndex, bucketIndex, placedIcon, null, iconBoxScale, iconPadding, iconAlongLine, true);
if (shapedIcon) {
iconQuads = addToBuffers ? getIconQuads(anchor, shapedIcon, iconBoxScale, line, layer, iconAlongLine, shapedText, globalProperties, featureProperties) : [];
iconCollisionFeature = new CollisionFeature(collisionBoxArray, line, anchor, featureIndex, sourceLayerIndex, bucketIndex, shapedIcon, null, iconBoxScale, iconPadding, iconAlongLine, true);
}

iconQuadStartIndex = this.symbolQuadsArray.length;
Expand Down
10 changes: 5 additions & 5 deletions js/symbol/quads.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -126,24 +126,24 @@ function getIconQuads(anchor, shapedIcon, boxScale, line, layer, alongLine, shap
* Create the quads used for rendering a text label.
*
* @param {Anchor} anchor
* @param {Shaping} shaping
* @param shapedText
* @param {number} boxScale A magic number for converting from glyph metric units to geometry units.
* @param {Array<Array<Point>>} line
* @param {StyleLayer} layer
* @param {boolean} alongLine Whether the label should be placed along the line.
* @returns {Array<SymbolQuad>}
* @private
*/
function getGlyphQuads(anchor, shaping, boxScale, line, layer, alongLine, verticalOrientation) {
function getGlyphQuads(anchor, shapedText, boxScale, line, layer, alongLine, verticalOrientation) {

var textRotate = layer.layout['text-rotate'] * Math.PI / 180;
var keepUpright = layer.layout['text-keep-upright'];

var placedGlyphs = shaping.placedGlyphs;
var shapedGlyphs = shapedText.shapedGlyphs;
var quads = [];

for (var k = 0; k < placedGlyphs.length; k++) {
var positionedGlyph = placedGlyphs[k];
for (var k = 0; k < shapedGlyphs.length; k++) {
var positionedGlyph = shapedGlyphs[k];
var glyph = positionedGlyph.glyph;
if (!glyph) continue;

Expand Down
0 js/symbol/place_icon.js → js/symbol/shape_icon.js
View file
Open in desktop
File renamed without changes.
87 changes: 42 additions & 45 deletions js/symbol/place_text.js → js/symbol/shape_text.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,10 @@ var newLine = 0x0a;

invisible[newLine] = breakable[newLine] = true;

module.exports = function placeText(text, glyphs, maxWidth, lineHeight, horizontalAlign, verticalAlign, justify, spacing, translate, verticalHeight, verticalOrientation) {
var placedGlyphs = [];
var placedText = {
placedGlyphs: placedGlyphs,
module.exports = function shapeText(text, glyphs, maxWidth, lineHeight, horizontalAlign, verticalAlign, justify, spacing, translate, verticalHeight, verticalOrientation) {
var shapedGlyphs = [];
var shapedText = {
shapedGlyphs: shapedGlyphs,
text: text,
top: translate[1],
bottom: translate[1],
Expand All @@ -75,7 +75,7 @@ module.exports = function placeText(text, glyphs, maxWidth, lineHeight, horizont

if (!glyph && codePoint !== newLine) continue;

placedGlyphs.push({codePoint: codePoint, x: x, y: y, glyph: glyph});
shapedGlyphs.push({codePoint: codePoint, x: x, y: y, glyph: glyph});

if (verticalOrientation) {
y += verticalHeight + spacing;
Expand All @@ -84,12 +84,12 @@ module.exports = function placeText(text, glyphs, maxWidth, lineHeight, horizont
}
}

if (!placedGlyphs.length) return false;
wrapTextLines(placedText, glyphs, lineHeight, maxWidth, horizontalAlign, verticalAlign, justify, translate, verticalHeight, verticalOrientation);
return placedText;
if (!shapedGlyphs.length) return false;
wrapTextLines(shapedText, glyphs, lineHeight, maxWidth, horizontalAlign, verticalAlign, justify, translate, verticalHeight, verticalOrientation);
return shapedText;
};

function wrapTextLines(placedText, glyphs, lineHeight, maxWidth, horizontalAlign, verticalAlign, justify, translate, verticalHeight, verticalOrientation) {
function wrapTextLines(shapedText, glyphs, lineHeight, maxWidth, horizontalAlign, verticalAlign, justify, translate, verticalHeight, verticalOrientation) {
var lastSafeBreak = null;

var lengthBeforeCurrentLine = 0;
Expand All @@ -98,36 +98,36 @@ function wrapTextLines(placedText, glyphs, lineHeight, maxWidth, horizontalAlign

var maxLineLength = 0;

var placedGlyphs = placedText.placedGlyphs;
var shapedGlyphs = shapedText.shapedGlyphs;

if (maxWidth) {

var wordLength = placedGlyphs.length;
var wordLength = shapedGlyphs.length;

for (var i = 0; i < placedGlyphs.length; i++) {
var positionedGlyph = placedGlyphs[i];
for (var i = 0; i < shapedGlyphs.length; i++) {
var shapedGlyph = shapedGlyphs[i];

positionedGlyph.x -= lengthBeforeCurrentLine;
positionedGlyph.y += lineHeight * line;
shapedGlyph.x -= lengthBeforeCurrentLine;
shapedGlyph.y += lineHeight * line;

if (positionedGlyph.x > maxWidth && lastSafeBreak !== null) {
if (shapedGlyph.x > maxWidth && lastSafeBreak !== null) {

var lineLength = placedGlyphs[lastSafeBreak + 1].x;
var lineLength = shapedGlyphs[lastSafeBreak + 1].x;
maxLineLength = Math.max(lineLength, maxLineLength);

for (var k = lastSafeBreak + 1; k <= i; k++) {
placedGlyphs[k].y += lineHeight;
placedGlyphs[k].x -= lineLength;
shapedGlyphs[k].y += lineHeight;
shapedGlyphs[k].x -= lineLength;
}

if (justify) {
// Collapse invisible characters.
var lineEnd = lastSafeBreak;
if (invisible[placedGlyphs[lastSafeBreak].codePoint]) {
if (invisible[shapedGlyphs[lastSafeBreak].codePoint]) {
lineEnd--;
}

justifyTextLine(placedGlyphs, glyphs, lineStartIndex, lineEnd, justify);
justifyTextLine(shapedGlyphs, glyphs, lineStartIndex, lineEnd, justify);
}

lineStartIndex = lastSafeBreak + 1;
Expand All @@ -136,11 +136,11 @@ function wrapTextLines(placedText, glyphs, lineHeight, maxWidth, horizontalAlign
line++;
}

if (placedGlyphs.length > 13) {
if (breakable[positionedGlyph.codePoint]) {
if (shapedGlyphs.length > 13) {
if (breakable[shapedGlyph.codePoint]) {
lastSafeBreak = i - 1;
}
if (!(breakable[positionedGlyph.codePoint]) && positionedGlyph.codePoint > 19968) {
if (!(breakable[shapedGlyph.codePoint]) && shapedGlyph.codePoint > 19968) {
lastSafeBreak = Math.round(wordLength / 3);
}
} else {
Expand All @@ -149,7 +149,7 @@ function wrapTextLines(placedText, glyphs, lineHeight, maxWidth, horizontalAlign
}
}

var lastPositionedGlyph = placedGlyphs[placedGlyphs.length - 1];
var lastPositionedGlyph = shapedGlyphs[shapedGlyphs.length - 1];

// For vertical labels, calculate 'length' along the y axis, and 'height' along the x axis
var axisPrimary = verticalOrientation ? 'y' : 'x';
Expand All @@ -161,32 +161,29 @@ function wrapTextLines(placedText, glyphs, lineHeight, maxWidth, horizontalAlign

var height = (line + 1) * leading;

justifyTextLine(placedGlyphs, glyphs, lineStartIndex, placedGlyphs.length - 1, justify);
align(placedGlyphs, justify, horizontalAlign, verticalAlign, maxLineLength, lineHeight, line, translate);
justifyTextLine(shapedGlyphs, glyphs, lineStartIndex, shapedGlyphs.length - 1, justify);

// align text?
var shiftX = (justify - horizontalAlign) * maxLineLength + translate[0];
var shiftY = (-verticalAlign * (line + 1) + 0.5) * lineHeight + translate[1];
for (var j = 0; j < shapedGlyphs.length; j++) {
shapedGlyphs[j].x += shiftX;
shapedGlyphs[j].y += shiftY;
}

// Calculate the bounding box
placedText.top += verticalOrientation ? -verticalAlign * maxLineLength : -verticalAlign * height;
placedText.bottom = verticalOrientation ? placedText.top + maxLineLength : placedText.top + height;
placedText.left += verticalOrientation ? -horizontalAlign * height : -horizontalAlign * maxLineLength;
placedText.right = verticalOrientation ? placedText.left + height : placedText.left + maxLineLength;
shapedText.top += verticalOrientation ? -verticalAlign * maxLineLength : -verticalAlign * height;
shapedText.bottom = verticalOrientation ? shapedText.top + maxLineLength : shapedText.top + height;
shapedText.left += verticalOrientation ? -horizontalAlign * height : -horizontalAlign * maxLineLength;
shapedText.right = verticalOrientation ? shapedText.left + height : shapedText.left + maxLineLength;
}

function justifyTextLine(placedGlyphs, glyphs, start, end, justify) {
var lastAdvance = glyphs[placedGlyphs[end].codePoint].advance;
var lineIndent = (placedGlyphs[end].x + lastAdvance) * justify;
function justifyTextLine(shapedGlyphs, glyphs, start, end, justify) {
var lastAdvance = glyphs[shapedGlyphs[end].codePoint].advance;
var lineIndent = (shapedGlyphs[end].x + lastAdvance) * justify;

for (var j = start; j <= end; j++) {
placedGlyphs[j].x -= lineIndent;
shapedGlyphs[j].x -= lineIndent;
}

}

function align(placedGlyphs, justify, horizontalAlign, verticalAlign, maxLineLength, lineHeight, line, translate) {
var shiftX = (justify - horizontalAlign) * maxLineLength + translate[0];
var shiftY = (-verticalAlign * (line + 1) + 0.5) * lineHeight + translate[1];

for (var j = 0; j < placedGlyphs.length; j++) {
placedGlyphs[j].x += shiftX;
placedGlyphs[j].y += shiftY;
}
}
2 changes: 1 addition & 1 deletion test/expected/text-shaping-default.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"placedGlyphs": [
"shapedGlyphs": [
{
"codePoint": 97,
"x": -32.5,
Expand Down
2 changes: 1 addition & 1 deletion test/expected/text-shaping-linebreak.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"placedGlyphs": [
"shapedGlyphs": [
{
"codePoint": 97,
"x": -32.5,
Expand Down
2 changes: 1 addition & 1 deletion test/expected/text-shaping-newline.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"placedGlyphs": [
"shapedGlyphs": [
{
"codePoint": 97,
"x": -32.5,
Expand Down
2 changes: 1 addition & 1 deletion test/expected/text-shaping-null.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"placedGlyphs": [
"shapedGlyphs": [
{
"codePoint": 104,
"x": -10,
Expand Down
2 changes: 1 addition & 1 deletion test/expected/text-shaping-spacing.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"placedGlyphs": [
"shapedGlyphs": [
{
"codePoint": 97,
"x": -38.5,
Expand Down
Loading