remapster

Map Pack 2

Hey all, time for another update!

We’ve been working hard on map pack 2 for the last 2 months and we’re finally ready to release! We also have made some fixes to the maps in map pack 1, driven by our own testing but most importantly by YOU. We’ve gotten lots of feedback from you all on fixes for maps and we’ve implemented them! Thank you for your help in bringing this vision to life. We really can’t stress enough how much your encouragements, suggestions, and fixes drive this project.

You can download the latest image files and game files (DATs) for pack 1 and pack 2 on the releases page of our github repository. The game files have been packaged such that they are not split into separate archives. Pack 1 and pack 2 are bundled together.

Installation

We *highly* recommend XIPivot for adding modded DAT files to the game. It’s an addon for Windower and Ashita both, and it’s a godsend for loading modded DATs into the game without worrying about game updates steamrolling them.

Head over to the XIPivot releases page and download the latest version for Windower or Ashita.
Extract the zip and copy the xipivot folder to your Windower or Ashita addons folder
Download the DATs from the remapster_maps releases page
Find the addon folder for XIPivot in Windower or Ashita.
Create a “data” folder and inside create a “DATs” folder. Once more, create a “remapster” folder inside the DATs folder.
Extract the DAT files so that the ROM folders are inside the remapster folder.
Create a settings.xml file in the “data” folder and add your remapster folder thus:
Load XIPivot in Windower or Ashita. The maps should load properly without needing to log out and load the game back up.

And that’s it! Enjoy!

Map Pack 1 zones

Aht Urhgan Whitegate	Escha – Ru’Aun	Ra’Kaznar Inner Court
Al Zahbi	Escha – Zi’Tah	Reisenjima
Bastok Markets	Foret de Hennetiel	Reisenjima Henge
Bastok Markets [S]	Kamihr Drifts	Rolanberry Fields
Bastok Mines	Konschtat Highlands	Rolanberry Fields [S]
Batallia Downs [S]	La Theine Plateau	Ru’Lude Gardens
Ceizak Battlegrounds	Lower Delkfutt’s Tower	Sauromugue Champaign
Chateau d’Oraguille	Lower Jeuno	Sih Gates
Cirdas Caverns	Marjami Ravine	Southern San d’Oria
Dho Gates	Metalworks	Southern San d’Oria [S]
Dynamis – Bastok	Mhaura	Tahrongi Canyon
Dynamis – Bastok [D]	Middle Delkfutt’s Tower	Tavnazian Safehold
Dynamis – Beaucedine	Moh Gates	Upper Delkfutt’s Tower
Dynamis – Buburimu	Morimar Basalt Fields	Upper Jeuno
Dynamis – Jeuno	Nashmau	Valkurm Dunes
Dynamis – Jeuno [D]	Norg	Walk of Echoes
Dynamis – Qufim	Northern San d’Oria	Walk of Echoes [P1]
Dynamis – San d’Oria	Nyzul Isle	Walk of Echoes [P2]
Dynamis – San d’Oria [D]	Outer Ra’Kaznar	Western Adoulin
Dynamis – Tavnazia	Port Bastok	Windurst Walls
Dynamis – Valkurm	Port Jeuno	Windurst Waters
Dynamis – Windurst	Port San d’Oria	Windurst Woods
Dynamis – Windurst [D]	Port Windurst	Woh Gates
Dynamis – Xarcabard	Pso’Xja	Yahse Hunting Grounds
Eastern Adoulin	Qufim Island	Yorcia Weald

Map Pack 2 zones

Abyssea – Altepa	Dangruf Wadi	Promyvion – Mea
Abyssea – Attohwa	Davoi	Promyvion – Vahzl
Abyssea – Konschtat	East Ronfaure	Qulun Dome
Abyssea – La Theine	East Ronfaure [S]	Ra’Kaznar Turris
Abyssea – Misareaux	East Sarutabaruta	Rala Waterways
Abyssea – Tahrongi	Fort Ghelsba	Riverne – Site #A01
Abyssea – Vunkerl	Garlaige Citadel	Riverne – Site #B01
Al’Taieu	Ghelsba Outpost	Ru’Aun Gardens
Altar Room	Giddeus	Sea Serpent Grotto
Alzadaal Undersea Ruins	Gusgen Mines	Selbina
Apollyon	Gustav Tunnel	South Gustaberg
Arrapago Reef	Inner Horutoto Ruins	Temenos
Attohwa Chasm	Jugner Forest	Temple of Uggalepih
Aydeewa Subterrane	Korroloka Tunnel	The Boyahda Tree
Batallia Downs	Kuftal Tunnel	The Eldieme Necropolis
Beadeaux	Lufaise Meadows	The Eldieme Necropolis [S]
Beaucedine Glacier	Mamook	The Sanctuary of Zi’Tah
Behemoth’s Dominion	Maze of Shakhrami	Uleguerand Range
Bhaflau Thickets	Meriphataud Mountains	Wajaom Woodlands
Bibiki Bay	Misareaux Coast	Waughroon Shrine
Bostaunieux Oubliette	Monastic Cavern	West Ronfaure
Buburimu Peninsula	Mount Zhayolm	West Sarutabaruta
Caedarva Mire	North Gustaberg	Xarcabard
Castle Oztroja	Palborough Mines	Yhoator Jungle
Castle Zvahl Baileys	Pashhow Marshlands	Yughott Grotto
Castle Zvahl Keep	Promyvion – Dem	Yuhtunga Jungle
Crawlers’ Nest	Promyvion – Holla	Zeruhn Mines

See you next time map masters,

Akaden

Mapping Attohwa Chasm

We’ve come across some wild stuff while working on this project, but nothing prepared me for just how challenging some of the maps were going to be to not only recreate (and stay true to the legacy map style), but to actually make them useful for players.

One of our biggest goals with Remapster is to make the absolute best and most accurate community versions of the zone maps that we can while still adhering to a style that pays homage to the reference material. Sure, it would have been easy and even faster to make vectorized, minimalistic, almost futuristic-looking versions of the maps. A lot of you would have probably been happy with that! But it defeated one of the core values of the project: to recreate the existing maps in high-definition and remain true to the character of FFXI.

So, what happens when you have an incredibly complex zone with an inversely simple map?

(ENTER ATTOHWA CHASM.)

I don’t think a zone exists in FFXI that has a starker contrast between its pathable terrain and its in-game map. I mean, look at this:

Let’s start with the most obvious part. You’ve got two large areas with a lot of low, bumpy hills pockmarking the landscape. What do you think as a player when you pop this bad boy open?

To me, at least, it reads like the terrain is bumpy, yet traversable. It’s still an open area, but you’re gonna be running up and down rolling hills most of the time. Is that how it actually is in-game?

NOPE.

Those wide open areas on the map? Yeah, they’re mazes. There are more mazes in this zone than in the Maze of Shakhrami, which has “maze” right in the title!

And how do you get from one side of this map to the other? Why, tunnels of course! But…

Hold on…

Where are the…

As you’ve probably surmised by now, none of the tunnels are visible or drawn. Now, let’s be fair, there are other maps in FFXI that have hidden tunnels; that’s not unique to Attohwa Chasm. But there are, like, a lot of them.

So how do we begin?

Well, thankfully, there are a few tools at our disposal. We are fortunate enough that there are still some websites online after all these years that are hosting some community maps that have tried to make sense of this absolute disaster of an area. Valiant efforts all, but hard to find any that truly capture the accuracy we want for Remapster.

What’s a video game cartographer to do? Why, tear open the video game, of course! Let’s get right at the guts – let’s rip that geometry mesh right out of the game files and really look at it closely.

Er, uh, okay. Um. Okay, this is… workable. At least I can see the tunnels more clearly! But there’s still the problem of those big spaces. How are we going to use this mess to get at our truly pathable terrain…?

The answer requires a bit of map surgery within our 3D suite.

I start by selecting all the polygon faces of the area I want to work on first. Let’s use the east side as our example.

Alright, great. Next, I wanna view this from a side view, and deselect polygons that a player character can walk on. We can get pretty darn close by drawing a selection fairly close to the “ground”.

That gets us roughly to here. Most of the polygons that rise above the ground plane are selected. Actually looks pretty cool…

I think you have probably guessed what’s next.

(presses Delete button)

Hot damn, now we’re getting somewhere! Everything in blue is, to the best of our ability, walkable! We might have missed a few small boulders, but in the end, that’s probably an acceptable level for Remapster purposes.

So we repeat this for the other side, and…

This is looking pretty helpful now! All that’s left to do is pull a screenshot of this into Photoshop and use it as a reference layer that we can trace.

After a few hours of drawing, we reach something like this:

Much more representative of the ground you can walk on, right? Now, it’s not an exact science, so we’re likely going to have a few spots in there that got missed due to terrain elevation and overhangs, etc., but we’re in really good shape!

So, to recap: by cross-referencing community drawn maps and manipulating the zone’s actual mesh geometry, we can get really, really close to polygon-perfect accuracy for zones that have really unhelpful maps.

Pretty neat, huh?

And yeah, we didn’t even come close to talking about the miasma streams that block certain pathways, or the narrow, precarious trail you have to take to the top of Parradamo Tor, but rest assured: Remapster’s got you covered there, too. 😉

Til’ next time!

-spalose

DAT Pack 1

Hey all, whew have we been busy!

Lots of discoveries abound after the release of Map Pack 1. And the community (yous guys) is so excited! This is *exactly* what we wanted to see. We can’t do this alone. We’ve got some plans to keep you all involved in the near future, so keep a look out on this page and elsewhere. I’ll talk about the details of the journey after I drop the guide and the files.

How to Install and Use the Map DATs

Head over to the XIPivot releases page and download the latest version for Windower or Ashita.
Extract the zip and copy the xipivot folder to your Windower or Ashita addons folder
Download the DATs from the remapster_maps releases page
Find the addon folder for XIPivot in Windower or Ashita.
Create a “data” folder and inside create a “DATs” folder. Once more, create a “remapster” folder inside the DATs folder.
Extract the DAT files so that the ROM folders are inside the remapster folder.
Create a settings.xml file in the “data” folder and add your remapster folder thus:
Load XIPivot in Windower or Ashita. The maps should load properly without needing to log out and load the game back up.

And that’s it! Enjoy!

The Juicy Details

Very recently, RadialArcana (FFXIAH) discovered how to update the in-game map DAT files to display a higher resolution than the base 512px. In his post, he teased it with a few images. Which of course got everyone excited. All at once, we got a whole bunch of private messages sending us the link. Kenshi sent me some details about how to do it manually and so I set off on a looooong quest to automate it. We’ve got a lot of maps to draw and I don’t want to sit here for days updating all the DAT files every time we push an update

My first task was to fix the tool (XI Tex Hammer) to write the necessary resolution to the right place. So I decompiled the source code and loaded it up in Visual Studio. None of the UI parts would compile for unimportant reasons. But I don’t want them anyway, so its chucked.

I managed to get the resolution saved properly and I added some command-line functionality and started off on the script to convert the images to DATs using my new old tool. Which worked!

Now I need a big long list of zone+map => DAT path, which as far as I can tell is not out there in a complete form. Most map packs focus on a few important maps. That’s not enough for us. I wrote a script to scour the DAT files for texture names that follow a certain pattern and then culled the outliers by hand. The list went from thousands to around 900 DATs. We know there’s less maps than that in the game, but some are repeated for various reasons. That doesn’t matter so much since our script will write them out in a second or two.

Alright, I’ve got all the DATs out, time to test and… it didn’t work for more than half the maps. This is when I discovered that most of the DATs store compressed images using DXT3. All I’ve been working with is palette-based files. ImageMagick doesn’t support DXT3. So I ~~scoured the internet~~ did a quick google search and found DDSTool which can convert png images to DDS compressed with DXT3 and it worked! Now I could import these DDS files into the DATs that need them compressed and I can live happily-ever-af–

Nevermind. I was about to push the fresh, crispy DATs out and in testing I discovered that the palette-based maps *didn’t* look so hot after all. You see, these have a limited color scope. And they don’t like how fresh our maps are. They looked beige. Which yes is most of the maps (by design) but we want the markers to *pop*. They’re supposed to be colorful and cute.

So back to ImageMagick to soften up our images. Fresh off the presses: 128 color versions of our maps. Gross. Well, they look better than beige-only maps, so that’s what we have. The game has a lot of limitations. Some can be overcome, some take extra effort. I’m certain we can find a way to make the maps look better, but for now this is the best we can offer.

Until next time, bit-cowboys,

Akaden

Map Pack 1

Hey everyone! We’ve had a lot of interest in Remapster in the last few weeks since our return announcement. While the Windower add-on is still deep in development, we’ve had a few requests for a downloadable map pack. So, the last few weeks, we took some time to prepare the very first map pack for Remapster. While this isn’t the originally how we intended to release these maps, we decided that it’d be the best way to get some more interest in the project and some feedback from the community while we work on more maps and more features for the add-on.

In this map pack, we have all of our “Priority 1” maps. We have a good variety of maps here, but we tried to prioritize some of the more frequently traveled areas of the game.

These maps are generated from the composite script we nicknamed “wiki”. We intended this to be a clean, overlay-free map with readable paths and topography while retaining a useful set of markers for normal adventuring. You can expect to see the following markers in these maps:

Homepoints, waypoints, survival guides, and escha portals
Shops, services, and guilds
Unity and Voidwatch rifts, and colonization and lair reives
Zone and map transition letters and numbers to denote connectivity
Climbing and watercraft launch points

We have some old classic zones, some new hotness, and even some maps that have never been drawn before. We had to delve deep into the zone geometry to get some of these maps. We hope you enjoy!

Below, you can download each pack in ~~512px,~~ 1024px and 2048px resolutions. We obviously recommend the crispiest 2048px resolution as it’s the most legible and it’s the native resolution that we draw the maps at. 1024px is a good compromise between the high fidelity and low bandwidth varieties. ~~512px is the only resolution able to be displayed in-game~~. Update: See the DAT file release post for instructions on how to get the 2048px maps into the game.

Download the 2048×2048 maps here!

Download the 1024×1024 maps here!

We’ll be keeping the github repository up to date and publishing releases there too. We’ll also be announcing major fixes and new map packs here.

If you have any feedback on the released maps, please submit an issue here.

-Akaden

Map Data, Margins, and Map Packs

Oh, my!

It’s been a minute. But, hey we’re back and we’re refreshed and we’re ready to work! We’ve got a few things to show off today to celebrate our first two weeks back from a hiatus.

Map Data

First up, an update on our progress in collecting data on maps. Over the past two weeks, we’ve been collecting as much as we can. We just passed the 40% mark on total data acquisition. It’s a little all over the place, but we do sort of have a priority system. This data includes:

Map ID/world coordinate bounds to image data
Transition icons (between zones and other maps) with a clickable action to change maps
Travel icons (home points, waypoints, etc)
Unity, Voidwatch, and ZNM spawns
Chests and Coffers
Gathering (logging, harvesting, and mining, as available)
and various others!

Another change to the map data is behind the scenes. We’ve moved from storing the data in a lua format to the JSON format. Lua was very convenient because we didn’t need a library to read the data. But with some of the features we’re planning, having map data in a more accessible format is important.

Margins

Okay! Boring text part is over, time for some visual goodies! We’ve added a new feature to the Megamap: edge margins. The map textures have the row/column identifiers on their own edges, but when the map is moved partially off-screen, they become occluded. This feature snaps them back into view!

These margins are built for style as well. The shadowreign, dynamis, escha, and abyssea maps all have a different style to differentiate them from present time maps, and so the intent is to have multiple sets of margin textures to layer over the other style maps to match their look. There’s a little bit of texture work to get this fully set up, but the code is ready for it.

Map Packs

One of the features we needed the JSON map data for was an ImageMagick script to generate usable map images with only the most important icons composited over. We created a list of icons that we thought were important to see, and cut out the clutter. Because the data is already there, we can generate as many maps as we want. Some examples might include: a treasure map with just coffers and chests, or a gathering map with just the gathering items.

Then we decided on a few resolutions that we thought would be useful to the community. We landed on 512×512 for compatibility with in-game maps, native 2048×2048 for the crispest, freshest maps available, and 1024×1024 for a very readable compromise suitable for wikis and other projects.

Here are some sample images to display the differences. These are not final, as we plan on checking the maps individually and adjusting icon placement and size to ensure the best readability possible.

Please look out for our first map pack announcement here. We have a bit of work to go on it, as there’s some logistics to manage. We still need to learn how to automate adding the maps to the DAT files, and we need to check each map for usability.

Well, that’s it for now. We’ll try to update more frequently. 2 years is a little bit of a stretch to wait for news.
– Akaden

Layers Like an Onion

One of the core features we’ve always planned for was the ability to hide the onslaught of icons in groups so you can focus on what’s important right now. Now that I have some time to sit down and write some code, it was what I wanted to focus on first.

We settled on Eastern Adoulin as our first map to put to the test. And it turned out to be quite the challenge. We quickly realized that the direction we were taking with the icons was a little optimistic. For the icons to live harmoniously onscreen with others, we were going to have to downsize. Spalose managed to redraw the important icons to be more consistent in size and read better at scale. I think they turned out beautiful.

The next step was placing these icons in a reusable way. Since we plan on making the maps available for wiki use once they’re complete, we’ll have to “bake” the maps into a single image. The goal is to bake them using the image data we use in the addon itself. We’ll have to write a script to do so, so the icon location has to be readable and easily applied to the baked map. So the icons are located in pixel space relative to the map’s maximum resolution.

Now that the icons are placed, we can show off the work all this was for. ZOOMING.

It’s amazing! You can see what’s going on by zooming in.

And now, the most important feature. The ability to hide groups of unwanted icons. We spent some time trying to come up with catch-all categories, and I don’t think we’re close to a full conclusion, but for now, these will do.

Who’s cutting onions? I’m not crying, you’re crying. Can we get a round of applause for those layer toggle buttons?

Next on the list is the ability to change maps easily. Quite a few zones have multiple maps that would be fun to cycle through. I guess it’d be useful too.

So long for now,
Akaden

Adoulin, Yorcia Weald, & Lots of Gates

It’s been a busy month, but that doesn’t mean I wasn’t able to find some downtime and crank out a few more maps!

I’m continuing to prioritize maps that see a lot of use by today’s playerbase, so I thought it would be a good time to tackle Western and Eastern Adoulin, as well as several of the Gates maps.

I also took a stab at actually making Yorcia Weald usable.

Caveats:

All of my maps so far are “base level” maps, meaning that aside from some text, pointers, and general terrain arrows, they’re pretty blank. I’m doing this primarily so that we can, in the addon, make all icon types able to be toggled on and off at the user’s discretion. A major issue I have with some community maps is that they are so cluttered it’s hard to make sense of them.
With regard to Yorcia Weald in particular – I did use a combination of several community-made maps to try and get the actual navigable areas represented in the illustration. But because those source maps were themselves pretty small, there may be some inaccuracies and I may have drawn a passage or wall where there isn’t one. Unfortunately, I don’t have a ton of time to also go scout the zones and make sure these are pixel-perfect. If you DO and you manage to find something inaccurate in my maps, please do not hesitate to let me know by leaving a comment!

On with the show!

Windurst & Lots More Icons

It’s been a bit of a slow/busy week, but I still managed to crank out the Windurst maps and started on Yorcia Weald (not showing that one just yet). Tackling an Adoulin field map also got me into a fun groove of trying some icons for colonization rieves.

The San d’Oria Suite and Yuhtunga Jungle

I wrapped up the San d’Orian collection of maps a day or so ago, and glad I did, because it was actually really boring and time-consuming to make. Turns out hundreds of vertical and diagonal lines aren’t fun to draw. Add that to list of things I don’t like about San d’Oria already. 😛

One thing you may note is that it’s missing home points, etc. As the addon develops, I’m drawing these items on separate layers. Current plan is for them to be toggled on and off at the player’s discretion, so what you’re seeing is the very “base” level map asset, and icons/markers will layer above it.

As a “break”, I tried out Yuhtunga Jungle because I wanted to tackle yet another proof of concept: a map with lots of underground tunnels. I gathered some community maps and tried to include everything I could that would help the most amount of people:

showing the tunnels
showing the zones it connects to that didn’t have labels before
showing one-directional cliff drops

Hope you’re all enjoying the progress so far!

The Giddeus Experiment

One of the goals of Remapster is to dig into the excellent community resources that have been created over the life of FFXI and utilize them to render extremely useful maps that don’t detract from the “legacy feel”.

I felt Giddeus was a great subject for pulling some of those resources into something that looked official, because it had a number of things that can create a frustrating game experience if you don’t have a good map:

One-way cliffs
Holes
Multiple maps
An exit to another area that was never clearly labled (Balga’s Dais)

Below, you’ll find the remastered versions of the Giddeus maps, labeled to help you travel between them:

As you can see, I utilized arrows and fonts that look intentional and communicate where the one-way cliff dropoffs are. I also clearly label Map #1 and Map #2, and where each of the various lettered paths take you.

I’m very much looking forward to remastering more of these kinds of zones in particular.