[DIP] WebView TTS

Recently there was an inquiry about whether our service could support TTS (Text To Speech). At the time there was nothing requiring direct action, so I didn't review the request in depth, but thinking it would be nice if the service supported TTS, I decided to do some additional research and write it up.

Web Speech API

The Web Speech API converts text into audio speech and plays it, using either a TTS engine built into the web browser or, on operating systems that allow access, the OS's voice engine.

The Web Speech API provides text-to-speech (SpeechSynthesis) and asynchronous speech recognition (SpeechRecognition).

SpeechSynthesis

• https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesis

This is the most essential API for TTS functionality, an interface that converts text into speech. Through simple methods such as speak, pause, and cancel, it takes text input and outputs it as speech, and can stop and remove it.

Available Properties

• paused: returns whether it is in a paused state (Boolean)
• pending: whether the utterance queue contains an utterance (Boolean)
• speaking: whether an utterance is in progress (Boolean)

Available Methods

• cancel: removes all utterances from the utterance queue
• getVoices: returns the list of all voices available on the device
• pause: puts it into a paused state
• resume: resumes from a paused state
• speak: adds an utterance to the utterance queue

In describing the Web Speech API, I said it takes text input and outputs it as speech. But looking at the description above, you'll see it talks about adding an utterance to a queue, speaking the utterance, and removing it. The Web Speech API doesn't simply take text and convert it to speech; it creates a separate object instance called an utterance (SpeechSynthesisUtterance) and processes the conversion to speech through it.

SpeechSynthesisUtterance

• https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesisUtterance

SpeechSynthesisUtterance is an object that holds the text to be spoken and properties about how it is output. It is the core object in TTS, injected into and processed by the Web Speech API's SpeechSynthesis.

Available Properties

• text: the text to be spoken
• lang: the language to use (ex. en-US, ko-KR)
• voice: the voice object to use (a voice provided by the browser)
• volume: volume (0.0 ~ 1.0, default: 1)
• rate: speaking speed (0.1 ~ 10, default: 1)
• pitch: pitch (0 ~ 2, default: 1)

Available Events

• onstart: called when speech playback starts
• onend: called when speech playback completes
• onerror: called when an error occurs during speech playback
• onpause: called when speech is paused
• onresume: called when resuming after a pause
• onmark: called when speech reaches the position of an SSML mark tag
• onboundary: called when a word or sentence boundary is reached

Now let's create a simple example that outputs text as speech using the two things described above, SpeechSynthesis and SpeechSynthesisUtterance.

tsx
["Hello", "Workd"].forEach((text) => {
  const utter = new SpeechSynthesisUtterance(text);
  utter.lang = "ko-KR";
  utter.volume = 1; // 0.0 ~ 1.0
  utter.rate = 1; // 0.1 ~ 10
  utter.pitch = 1; // 0 ~ 2
  utter.onstart = () => console.log(`Start: ${text}`);
  utter.onend = () => console.log(`End: ${text}`);
  utter.onerror = (e) => console.log(`Error: ${e}`);
  speechSynthesis.speak(utter);
});

Why Is Event Handling Done in SpeechSynthesisUtterance?

While researching, I grew curious about why events are managed in SpeechSynthesisUtterance rather than in SpeechSynthesis, which has the methods to start and pause the actual speech output, so I looked into it.

The reason is that in TTS, the role of SpeechSynthesis is to act as a controller that manages and controls the queue of utterances. SpeechSynthesisUtterance, on the other hand, is an object that holds the text to be spoken and its properties, with the individual state of each utterance.

Because each individual utterance needs to be designed so that event handling can be done separately according to its own properties or state, the handling of events for an utterance's start, end, or pause is managed in SpeechSynthesisUtterance.

The reason I was curious about this is that, comparing it to React, I thought of SpeechSynthesis as the component that handles rendering, and SpeechSynthesisUtterance as the state managed within the component. When I first saw the Web Speech API and imagined grafting this design onto React, I pictured a form where events of a component are managed from within its state, which is what sparked my curiosity.

*On second thought, within React's massive rendering logic, a component can also be viewed as a state within it. If you think of it as events being managed within the state that is the component managed inside the rendering logic, it seems it could be seen as the same design.

Lastly, while wrapping up the Web Speech API, unfortunately I confirmed that SpeechSynthesis, which outputs text as speech, and SpeechSynthesisUtterance, which manages utterance information, are not supported in the Android WebView, both needed to implement TTS.

Google Cloud Text to Speech API

• https://www.npmjs.com/package/@google-cloud/text-to-speech

Another way to implement TTS is to generate Audio Content on the server based on the text passed from the client, then output the audio from the Audio Content returned to the client.

Node.js Server

tsx
// index.js
import express from "express";
import textToSpeech from "@google-cloud/text-to-speech";
import dotenv from "dotenv";
import fs from "fs/promises";

dotenv.config();

const app = express();
const port = 3000;

// GCP authentication setup
const client = new textToSpeech.TextToSpeechClient({
  keyFilename: "./google-credentials.json",
});

app.get("/api/tts", async (req, res) => {
  const text = req.query.text;

  if (!text) {
    return res.status(400).send("text parameter is required");
  }

  try {
    const [response] = await client.synthesizeSpeech({
      input: { text },
      voice: {
        languageCode: "ko-KR",
        ssmlGender: "FEMALE", // male: MALE, neutral: NEUTRAL
      },
      audioConfig: {
        audioEncoding: "MP3",
      },
    });

    res.setHeader("Content-Type", "audio/mpeg");
    res.send(response.audioContent);
  } catch (err) {
    console.error(err);
    res.status(500).send("TTS failed");
  }
});

app.listen(port, () => {
  console.log(`TTS server running on http://localhost:${port}`);
});

WebView JavaScript

tsx
const speak = async (text) => {
  const audio = new Audio(`/api/tts?text=${encodeURIComponent(text)}`);
  await audio.play();
};

The approach above using the Google Cloud Text to Speech API additionally requires creating a project in Google Cloud and setting up authentication, and although over 4 million characters per month are free, it requires payment beyond that. If actually adopted in a service, 4 million characters is a character count that would be consumed quickly, so I think using it for real service application would not be easy.

say & espeak TTS

Since the Google Cloud Text to Speech API seemed hard to use due to its pricing policy, I looked for free TTS modules and found a few. Among them, I looked into the say module and the espeak module supported in a Node.js environment.

say.js

The say module internally generates an audio-generation CLI suited to the running system OS and uses the OS's built-in TTS system to generate audio speech from text.

The OSes handled in the say module are darwin, linux, and win32. It was implemented to generate a CLI to convert text to speech using the command for each OS: say for darwin, festival for linux, and powershell for win32.

In fact, in a macOS terminal environment, you can easily confirm that speech is output by typing the following.

$ say “안녕하세요”

Below are options you can set with say; what surprised me was that you can specify the voice to use. You can check the voices supported in the current environment as a list, and the default Korean voice support is set to “Yuna”, which outputs the speech.

$ say -v “?”

Below is simple example code that converts text to audio speech and responds on a Node.js server built with express. As mentioned earlier, in a macOS environment you can use the say command directly as a CLI, so it can be implemented by running the command without installing a module.

Node.js Server

tsx
// server.js
const express = require("express");
const { exec } = require("child_process");
const fs = require("fs");

const app = express();
const port = 3000;

app.get("/tts", (req, res) => {
  const text = req.query.text;
  const filePath = "output.aiff";

  if (!text) {
    return res.status(400).send("Missing text query parameter");
  }

  // generate an audio file with the say command
  exec(`say -o ${filePath} "${text}"`, (err) => {
    if (err) {
      console.error("TTS generation failed:", err);
      return res.status(500).send("TTS failed");
    }

    // stream the generated file as the response
    res.setHeader("Content-Type", "audio/aiff");
    const stream = fs.createReadStream(filePath);
    stream.pipe(res);

    // delete the file after streaming ends
    stream.on("close", () => {
      fs.unlinkSync(filePath);
    });
  });
});

app.listen(port, () => {
  console.log(
    `✅ TTS server running: http://localhost:${port}/tts?text=안녕하세요`,
  );
});

espeak

Next, espeak was implemented, like say, to build a command to generate audio speech data, and since the implementation itself was so simple, it seemed better to implement it directly with the command rather than using the module.

espeak can be used via CLI on various platforms such as Linux, Windows, and macOS. However, I could confirm that its quality is much lower than the audio speech data generated by say on macOS.

If you check with the command below, you can confirm that a very awkward, robot-like voice is output.

$ espeak “안녕하세요”

Below are the options supported in espeak; when checked with voices, you can see the list of supported voices. However, perhaps because Korean isn't included in the default list, it outputs a voice like a foreigner robot speaking Korean, while when using English it felt like the output wasn't too bad.

espeak can also generate audio speech data from text via the command, and it can be implemented simply in code as shown below.

Node.js Server

tsx
// server.js
const express = require("express");
const { exec } = require("child_process");
const fs = require("fs");
const path = require("path");
const { randomUUID } = require("crypto");

const app = express();
const port = 3000;

app.get("/tts", (req, res) => {
  const text = req.query.text;
  if (!text) return res.status(400).send('Missing "text" query parameter');

  const filename = `espeak-${randomUUID()}.wav`;
  const filePath = path.join(__dirname, filename);

  // generate a wav file with espeak
  const command = `espeak -v ko -w ${filePath} "${text}"`;

  exec(command, (err) => {
    if (err) {
      console.error("espeak error:", err);
      return res.status(500).send("TTS failed");
    }

    // respond with the file
    res.setHeader("Content-Type", "audio/wav");
    const stream = fs.createReadStream(filePath);
    stream.pipe(res);

    // delete the temp file after the response
    stream.on("close", () => {
      fs.unlink(filePath, (err) => {
        if (err) console.warn("Temp file cleanup failed:", err);
      });
    });
  });
});

app.listen(port, () => {
  console.log(
    `✅ TTS server running: http://localhost:${port}/tts?text=안녕하세요`,
  );
});

Researching the materials above, I found that the Web Speech API has environment-specific constraints, the Google Cloud Text to Speech API has its pricing policy, and the methods using say and espeak require server-side processing and have quality issues depending on the platform (OS). For these reasons, I concluded that implementing TTS on the FE itself to support it in the service would not be easy, so I did additional research on ways to leverage features implemented on the client in the FE.

Android & iOS TTS

Android ↔ WebView Bridge

• https://developer.android.com/reference/android/speech/tts/TextToSpeech

The first method is to implement text-to-speech output using Android's built-in TextToSpeech API and provide that feature inside the WebView as a bridge function.

Android Kotlin

tsx
class MainActivity : AppCompatActivity(), TextToSpeech.OnInitListener {

    private lateinit var webView: WebView
    private lateinit var tts: TextToSpeech

    override fun onCreate(savedInstanceState: Bundle?) {
        super.onCreate(savedInstanceState)

        webView = WebView(this)
        setContentView(webView)

        // initialize TTS
        tts = TextToSpeech(this, this)

        // allow JavaScript
        webView.settings.javaScriptEnabled = true

        // connect the JS <-> Android bridge
        webView.addJavascriptInterface(JSBridge(), "**AndroidBridge**")

        // load the web page
        webView.loadUrl("file:///android_asset/index.html")
    }

    override fun onInit(status: Int) {
        if (status == TextToSpeech.SUCCESS) {
            tts.language = Locale.KOREAN
        }
    }

    inner class JSBridge {
        @JavascriptInterface
        fun speak(text: String) {
            Log.d("JSBridge", "Speaking: $text")
            tts.speak(text, TextToSpeech.QUEUE_FLUSH, null, null)
        }
    }

    override fun onDestroy() {
        tts.shutdown()
        super.onDestroy()
    }
}

WebView JavaScript

tsx
// Android
function speak() {
  const text = document.getElementById("text").value;
  if (window.AndroidBridge?.speak) {
    window.AndroidBridge.speak(text);
  } else {
    alert("The Android TTS bridge is not connected.");
  }
}

iOS ↔ WebView Bridge

• https://developer.apple.com/documentation/avfaudio/avspeechsynthesisvoice
• https://developer.apple.com/documentation/avfaudio/avspeechutterance

On iOS, TTS can be handled through the utterance object AVSpeechUtterance and the AVSpeechSynthesisVoice object responsible for the voice to use.

iOS Swift

swift
import UIKit
import WebKit
import AVFoundation

class ViewController: UIViewController, WKScriptMessageHandler {
    var webView: WKWebView!
    let speechSynthesizer = AVSpeechSynthesizer()

    override func viewDidLoad() {
        super.viewDidLoad()

        // 1. Set up the JS -> Native bridge
        let contentController = WKUserContentController()
        contentController.add(self, name: "speak")

        // 2. Configure the WebView
        let config = WKWebViewConfiguration()
        config.userContentController = contentController

        webView = WKWebView(frame: self.view.bounds, configuration: config)
        self.view.addSubview(webView)

        // 3. Load the web page
        if let url = Bundle.main.url(forResource: "index", withExtension: "html") {
            webView.loadFileURL(url, allowingReadAccessTo: url.deletingLastPathComponent())
        }
    }

    // 4. Handle JS messages (handler)
    func userContentController(_ userContentController: WKUserContentController, didReceive message: WKScriptMessage) {
        if message.name == "speak", let text = message.body as? String {
            let utterance = AVSpeechUtterance(string: text)
            utterance.voice = AVSpeechSynthesisVoice(language: "ko-KR")
            speechSynthesizer.speak(utterance)
        }
    }
}

WebView JavaScript

tsx
// iOS
function speak() {
  const text = document.getElementById("text").value;
  if (window.webkit?.messageHandlers?.speak) {
    window.webkit.messageHandlers.speak.postMessage(text);
  } else {
    alert("The iOS bridge is not connected.");
  }
}

Why Does Android Manage Utterance Settings on the TTS Engine Instance?

Unlike the Web Speech API's design approach, Android is designed so that utterance properties are set on the TTS engine context.

Android's TTS engine is registered as a system service and has a structure shared by all apps/components. A single TTS engine runs in the system, and even if an app creates multiple TTS instances, they all use the same engine. This structure has the advantages of saving system resources and being usable with consistent user settings. Also, the Android TTS engine operates as a singleton service and is efficient for reuse in that a single configuration maintains the same settings across subsequent utterances.

On iOS, in line with Apple's object-oriented API design philosophy, the setting state is managed on the utterance itself, the same as the Web Speech API, and it is processed with various utterance configurations. Additionally, the iOS TTS engine is not a singleton service but can operate independently, so it's said to be designed to allow various utterances to be used flexibly.

In Conclusion

I looked into several ways to implement TTS inside a WebView. As smartphones have become widespread, many services seem to be developed on a WebView basis.

There aren't many smartphone OSes, but when you actually handle per-OS processing, situations arise where you need the client's help, and in that process I sometimes see code where dependencies grow. Each time this happens, I'm reminded of the days developing in a PC browser environment. Those days of being annoyed trying to match browser compatibility across IE, Chrome, Safari, and more.

Now, thanks to the flow of the times and technological progress, the development environment has shifted from PC browsers to smartphones. If so, I'm curious and looking forward to imagining what kind of environment we frontend developers will face next.

"It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change."

- Charles Darwin -