StripMaker: Perception-driven Learned Vector Sketch Consolidation

Given a vector sketch with multiple overdrawn strokes (a) StripMaker automatically consolidates it (f) replacing each detected viewer perceived strip of strokes (e, each strip in different color) with the corresponding intended curve. StripMaker outputs (e) are better aligned with user expectations than those produced by state-of-the-art algorithmic alternatives (b,c,d). Inset in (d) shows [Liu et al. 2018] strips. Frames point to artifacts in outputs of previous methods. Input image © jwalsh under CC-BY-2.0.

Abstract

Artist sketches often use multiple overdrawn strokes to depict a single intended curve. Humans effortlessly mentally consolidate such sketches by detecting groups of overdrawn strokes and replacing them with the corresponding intended curves. While this mental process is near instantaneous, manually annotating or retracing sketches to communicate this intended mental image is highly time consuming; yet most sketch applications are not designed to handle overdrawing and can only operate on overdrawing-free, consolidated sketches. We propose StripMaker, a new and robust learning based method for automatic consolidation of raw vector sketches. We avoid the need for an unsustainably large manually annotated learning corpus by leveraging observations about artist workflow and perceptual cues viewers employ when mentally consolidating sketches. We train two perception-aware classifiers that assess the likelihood that a pair of stroke groups jointly depicts the same intended curve: our first classifier is purely local and only accounts for the properties of the evaluated strokes; our second classifier incorporates global context and is designed to operate on approximately consolidated sketches. We embed these classifiers within a consolidation framework that leverages artist workflow: we first process strokes in the order they were drawn and use our local classifier to arrive at an approximate consolidation output, then use the contextual classifier to refine this output and finalize the consolidated result. We validate StripMaker by comparing its results to manual consolidation outputs and algorithmic alternatives. StripMaker achieves comparable performance to manual consolidation. In a comparative study participants preferred our results by a 53% margin over those of the closest algorithmic alternative (67% versus 14%, other/neither 19%).

Paper

Downloads

Additional Results

Additional comparisons with raster-space consolidation methods:

Additional comparisons with simultaneous consolidation and vectorization methods:

Additional comparisons with the state of the art vector consolidation approach of [Liu et al. 2018]:

Consolidation applications:

Applying topology cleanup [Yin et al. 2022] directly to a typical input (a) produces numerous undesirable tiny regions (545 on this input) (b). Consolidating the input with our method produces the viewer expected topology facilitating colorization (d). Our output strips (f) facilitate per-strip manipulations, such as gradient-based recolorization with gradient (g). Left input image © Preston Blair. Right input image © Maria Fiddler (aka Maria Hegedus) under CC-BY-NC-SA-4.0.

Video

BiBTeX

@article{stripmaker,
  title = {StripMaker: Perception-Driven Learned Vector Sketch Consolidation},
  author = {Liu, Chenxi and Aoki, Toshiki and Bessmeltsev, Mikhail and Sheffer, Alla},
  year = {2023},
  issue_date = {August 2023},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  volume = {42},
  number = {4},
  journal = {ACM Trans. Graph.},
  month = {jul},
  articleno = {55},
  numpages = {15},
  issn = {0730-0301},
  url = {https://doi.org/10.1145/3592130},
  doi = {10.1145/3592130}
}