Porting MYTH, Part I: Introduction

Part1Hey, everyone. BlackDragonHunt here—part-time translator, part-time code monkey—to talk a little bit about what went into porting MYTH. This is part one in a three-part series.

NScripter

To start with a little background, MYTH was originally scripted in NScripter, a visual novel engine written by Naoki Takahashi. NScripter has been around since 1999, and if you’re familiar with visual novels, chances are you’ve at least heard of (if not played) some games that use the engine—such as Higurashi, Umineko, Narcissu, and Tsukihime, just to name a few.

NScripter code resembles a mix of BASIC and assembly language, equipped with pretty much all the basic functionality you’d expect of a visual novel engine. For example, here’s a short excerpt from the beginning of MYTH:

*R1_ASGA00
gosub *windowset_nv
mov %chapter,2
mov $now,"違世界プロローグ「意識の始点」"
if %flag_H == 1 skip 2
if %1005 == 1 && lchk *R1_ASGA00_ok skip 2
goto *R1_ASGA00_top
mov %300,1
name "シーンスキップ",0
mov $52,"*R1_ASGA00_ok"
mov $54,"*R1_ASGA00_top"
goto *sceneskipcheck
*R1_ASGA00_top
mov %300,0
bg "black",18
bgm "sound\se\tokei.wav"
HB1
br:br:br:br:br
 暗い、黒い闇にいた。\
br:br:br:br:br
 そこを『黒』とも『闇』とも認識出来るだけ、
 僕は正常。@
 他にも『時計』の『音』だって、分かる。@
 『階下』という言葉、『足音』という言葉だって。@
 『ママ』という言葉だって。@
br
 分かるんだ。\

The results of the above script running in NScripter.

This code:

  • creates a label
  • calls a function that sets up the text box
  • sets a couple bookkeeping variables
  • checks to see if you’ve already read the scene, offering to skip it entirely if you have
  • sets the background to the color black
  • starts playing BGM
  • displays the text box
  • and finally, shows two screens of text

It’s relatively straightforward, if a bit verbose—in total, the entire game is a little over 90,000 lines of NScripter code.

NScripter and English

As with many visual novel engines, NScripter is designed primarily for use with Japanese text—scripts must be encoded in Shift-JIS, which for all intents and purposes restricts us to the use of the ASCII range, and it doesn’t support proportional fonts. So as-is, NScripter isn’t ideal for use in an English-translated game.

There are other options, though: ONScripter, an open-source clone of NScripter; ONScripter-EN, a branch of ONScripter tuned for use with English text; and Ponscripter, or “Proportional ONScripter,” which has support for Unicode and (as the name implies) proportional fonts. The farther down the line you go, the nicer they are for English, but the less compatible they are with the original NScripter scripts.

I experimented with each of the above options before tentatively settling on Ponscripter-fork, to my knowledge the most up-to-date version of the Ponscripter engine (with the added benefit of Steam support built-in). Nothing was able to handle the MYTH script as-is, but Ponscripter-fork looked like the most promising option in terms of providing a pleasant user experience.

So I pulled down a copy of the code and started chipping away at it, attempting to implement any missing commands needed to get the MYTH script into a playable condition. But after a couple days of hacking, it became clear that both the script and the engine were going to need a lot more work than originally anticipated to get them to play nicely. In addition to the missing built-in NScripter functionality, MYTH’s intricate UI brought Ponscripter to its knees with frequent crashes and images that would go missing for no apparent reason. But perhaps the biggest obstacle was that Ponscripter doesn’t support plugins, which MYTH makes fairly considerable use of, meaning heavy rewrites would be necessary.

MYTH trying its best to run in Ponscripter.

MYTH trying its best to run in Ponscripter.

It was around this time that it was decided that our release of MYTH would include the Japanese text, which was yet another hurdle I had no idea how to approach using Ponscripter. (Witch Hunt would later prove themselves much cleverer than me, managing to implement both text swapping and image swapping into Ponscripter for the MangaGamer release of Umineko.) So I decided this approach was going to be too much work and took a cue from Doddler, choosing instead to port the game to a new engine entirely. Rather than using Unity, as he does, I decided I wanted to use Ren’Py. Python is my language of choice, and I’d always wanted to try building a game in the engine.

Ren’Py

Ren’Py is a free, open-source visual novel engine developed primarily by Tom “PyTom” Rothamel. Ren’Py has been used for a wide variety of games, including our own Demon Master Chris and A Kiss For The Petals – Remembering How We Met (which was also ported from another language).

The engine runs on Python, so Ren’Py script naturally looks and feels a whole lot like Python (it even lets you use Python code directly). It has proper Unicode support, built-in functionality for switching between multiple languages, multi-platform, Steam integration, and is extremely flexible and extendable (which I’ll be taking full, horrifying advantage of later).

Here’s the same block of NScripter code from above, automatically transpiled into Ren’Py code:

label R1_ASGA00:
  windowset "nv"
  $ nums[chapter] = 2
  $ strs[now] = "違世界プロローグ「意識の始点」"
  if nums[flag_H] == 1:
    jump _R1_ASGA00__00006
  if nums[1005] == 1 and renpy.seen_label("R1_ASGA00_ok"):
    jump _R1_ASGA00__00007
label _R1_ASGA00__00006:
  jump R1_ASGA00_top
label _R1_ASGA00__00007:
  $ nums[300] = 1
  set_name "シーンスキップ" 0
  $ strs[52] = "*R1_ASGA00_ok"
  $ strs[54] = "*R1_ASGA00_top"
  jump sceneskipcheck
label R1_ASGA00_top:
  $ nums[300] = 0
  scene bg black
  with effect18
  clear_sprite 0
  play music "sound/se/tokei.wav" fadeout fadeout_time fadein fadein_time loop
  window show effect12
  translate None _R1_ASGA00_line_00000:
    spk "\n\n\n\n\n 暗い、黒い闇にいた。"
  translate None _R1_ASGA00_line_00001:
    spk "\n\n\n\n\n そこを『黒』とも『闇』とも認識出来るだけ、\n 僕は正常。{w}\n 他にも『時計』の『音』だって、分かる。{w}\n 『階下』という言葉、『足音』という言葉だって。{w}\n 『ママ』という言葉だって。{w}\n\n 分かるんだ。"
MYTH running in Ren'Py.]

MYTH running in Ren’Py.

Ignoring the rather ugly scaffolding code necessary to emulate some of NScripter’s low-level functionality, it’s fairly easy to see that this is doing essentially the same thing as the excerpt above. It’s also a lot easier to read—though still a far cry from how you’d probably write it by hand.

So with an engine finally picked out and feeling a whole lot more comfortable, being on my home turf, I set out on the (still extremely daunting) task of converting all 90,000+ lines of NScripter code into (nearly) identically functional Ren’Py code.

Breaking Things Down

Broadly speaking, there are a couple different ways you can go about porting a VN to a new engine. You can, as Doddler does, write code that parses and runs the original script directly—essentially a single-game emulator of sorts—or you can translate the code into a new (presumably existing) language. In choosing Ren’Py for my port, I did the latter.

As I mentioned above, NScripter code resembles a mix of BASIC and assembly language, while Ren’Py is based in Python. You can probably imagine the two look almost nothing alike, and there are some fundamental differences in the way the two engines operate that make 1:1 translation nearly impossible. So first, I had to divide the 90,000 line script up into two parts: a “program” section and a “script” section.

The program section consists of the various subroutines responsible for implementing game-specific functionality such as menus, the user interface, and handling the general flow of the game. These are essentially the game’s skeleton, amounting to ~7,500 lines of code, which I would have to rebuild by hand in Ren’Py.

The script section, on the other hand, is the meat of the game—it’s where all the text is contained, where sprites are shown and music is played. This section, just shy of 83,000 lines of code, is structured fairly regularly and generally doesn’t do anything too complex or unpredictable. All the fancy stuff is off in the program section, and the script simply calls those routines as needed. This section I would attempt to transpile into Ren’Py automatically, as doing it by hand would be a herculean task.

For these posts, I’ll mostly be focusing on the steps required to make the transpilation step possible.

The Easy Part

After breaking the script up into logical sections, I set out to write a preliminary transpilation script for automatically converting a subset of NScripter code into Ren’Py code.

As both are engines designed for creating visual novels, a decent chunk of basic NScripter functionality has fairly straightforward Ren’Py equivalents. A line prefixed with an asterisk creates a label. “mov” assigns a variable. “goto” means “jump”. “bgm” means “play music”. “return” means… “return”.

Not everything was so straightforward, but with the basics implemented, I was able to have something resembling a game up and running in just a couple days. But “something resembling a game” isn’t good enough—I had to replicate everything, not just the easy stuff—and I had a long way to go.

Data Management

All but the most basic of games inevitably have to store some data somewhere, so VN engines offer you variables where you can store that data. Broadly speaking, there are two different types of data a game could want to keep track of: local, such as flags or affection points, which are specific to an individual playthrough; and global, such as whether you’ve completed certain routes or unlocked the extras menu.

Ren’Py allows you to store local data in standard, named Python variables, and global data is stored on a special object called “persistent“.

NScripter, meanwhile, offers two main types of variables—numbers and strings (there are also arrays, but MYTH doesn’t use them outside the UI, so we’ll gloss over that). Variables don’t have names, but are rather stored in a 4096-element list. They can be accessed using numerical literals, named constants defined at the beginning of the script or, indeed, the value stored in another numeric variable. NScripter distinguishes between local and global data by its position in the list—values with an index of less than 500 are local, and the rest are global.

For example: %10 is the numeric value at index 10. $now is the string value at the index designated by the constant “now”. $%0 is the string value at the index designated by the value stored in index 0 of the numeric list.

To replicate this functionality in Ren’Py, I needed to do two things: first, create lists in which to store these numbers and strings. Second, I had to figure out some way to determine whether any given variable was accessing data in the local or global range.

Initially, I tried doing this in the transpilation step, parsing for named constants and checking to see whether it was in the global range, then substituting in the appropriate Ren’Py list. However, because NScripter allows for indirect indexing using the value of other variables, this approach was ultimately infeasible without keeping track of a huge amount of state.

So instead, I took it from a different angle, creating a simple stand-in class that does nothing but forward accesses on it to the appropriate local or global list. This way, I didn’t have to care what any variable accesses looked like, as the game itself would handle it all for me.

class NScripterNum:
  def __getitem__(self, key):
    if key < GLOBAL_START:
      return store._nums[key]
    else:
      return persistent._nums[key]
  
  def __setitem__(self, key, value):
    if key < GLOBAL_START:
      store._nums[key] = value
    else:
      persistent._nums[key] = value

Anyway, I think that’s probably enough for an introductory post. I hope you’re ready, because things are going to start getting a whole lot messier from here.


Part II: Doing Very, Very Naughty Things! | Part III: How Not to Code a Game in Ren’Py

Bookmark the permalink.

One Comment

  1. I know nothing regarding programming, but this was an intriguing read. Keep it up.

Leave a Reply