Artificial intelligence Web Development

Convert Sketch to HTML Code using Sketch2Code AI

Written by Gaurang Sharad

Have you ever pondered that you can transform your whiteboard sketches to HTML code! Yes, it is possible now. Microsoft has created a new AI platform that is easy to use and turns your Sketch to HTML codes. All you need is a hand-drawn image, and it generates the HTML out of it. So no more coding, tinkering, or delays in your projects. Let’s get started.

If you are a designer, then you must be familiar with the whiteboard and your pieces of sketch playlets. Microsoft has now come up with a new AI concept where a computer vision model understands your paper drawn designs and converts them directly to the HTML code. This helps all the collaboratives of a design team to share ideas on a single platform with the respective HTML code. Sounds great right! Let’s dig deep into this in-tech.

What is Sketch2Code and How it Works?

Microsoft has always been a great innovator that believes in simplifying the efforts of its users. Sketch to Code is also a great example of their innovations as it becomes a cakewalk for the designers to develop HTML code for their designs. But many things take place behind the scenario to make it possible.

Sketch2Code

The Custom Vision Model or AI model is trained with millions of images and graphs that enable it to detect the objects of your whiteboard. So, whenever you draw on the whiteboard or upload a copy of your sketch, a Windows app takes a picture and submit that image to the AI frame. The prebuilt AI model then executes the image and tries to detect all the objects or elements drawn like buttons, boxes, text, combo box, etc. Later it generates an HTML or any other markup language code for the design. Then it gets displayed in the application. Users are free to use this service, and it reduces the delay time for the designers that can take place in manual conversion of requirements to code.
Sketch2Code

The Paradigm that Sketch2Code Follows

Many things take place when a user scribbles or draws a pattern on the Whiteboard. Here, you can go through the workflow and work behind this web-based application that how it transforms a sketch to HTML code.

  1. Wireframes
    On this level, the user draws the design on a whiteboard that is intended to be converted into HTML, or users can simply upload a picture of their sketch drawn on paper to the UI.
  2. Pictures
    A Windows app then takes a photo of the sketch or image and shares it with the AI model for identification.
  3. Application
    The Custom Vision and Computer Vision APIs then do the object detection and try to match the patterns with the stored images in Azure Blob Storage using the algorithm. After the identification of the objects and elements like buttons, boxes, etc. the algorithm tries to recognize texts and handwriting using the OCR(Optical Character Recognition).
  4. Markup Generation
    You can see all the application layer or algorithm results in the web application. Now, users can extract the code as HTML, XAML, or in the form of UWP(Universal Windows Platform).

After the conversion of the sketch to code, the user can extract or download the markup code and can make it work for their situation.

How to Convert Sketch to HTML Code using Microsoft’s AI?

Microsoft Sketch2Code is a web-based application that can be easily used in just three simple steps:

    1. Upload the Design
      Open the web application and click on the Upload Design button, if the sketch is on the working system. Otherwise, click on Take a Picture button and capture the image of your design sketch to be converted.
      Sketch2Code
    2. Sketch2Code is at Work!
      After uploading the image, the Custom Vision and Computer Vision APIs start to predict the elements present in the sketch, and OCR looks for the text available. Then they create the HTML code out of it.
    3. Download Your HTML
      Now, you can take a preview of the outcome, and you can also see all the Predicted Objects Details. Click on the Download Your HTML Code button, and all the code will get downloaded in a flick.
      Sketch2Code

Microsoft Sketch2Code is a great invention to overcome designing issues, and lets you remove the delay of designing the layouts. It’s free to use application that comes with dynamic designing. So, designers can scribble their ideas on paper or whiteboard and can test the results in no time. This method made it easy and convenient to convert sketch to code. I hope you found it helpful.

About the author

Gaurang Sharad

Leave a Comment