[UE4] Analyzing your game textures streaming using Python and a CSV file

Hi guys, today I would like to share with you a python script I wrote to order a CSV extracted by the ListStreamingTextures command in Unreal Engine 4.
I used this in Die Young production to analyze my streaming textures log, since the UE4 command just listed textures without caring about any order rule like size, wanted size and so on.

This was really useful to me to check that textures were always streaming correctly where needed at the size I wanted them to avoid video memory waste.

The CSV extractor was made by my colleague Vladimir Ivanov at Indiegala in C++ editing the command class.

Once I had the CSV extracted, I started analyzing it an what I did was basically applying my Python knowledge to order it.

The result of the command at the beginning of the game in Editor gave me this 01.JPG

A lot of messy stuff.

Textures returned by the console command are by default ordered by name, which is not so useful when you are trying to understand which are the heavy ones or the wrong loaded ones.

So this is the plugin I wrote in python:


import csv
import operator

import Tkinter as tk
from Tkinter import *

from tkMessageBox import *
from tkFileDialog import *

#Define the list of the texture groups to use as filter
textureFilterChoices = ['World',
           'WorldNormalMap',
           'WorldSpecular',
           'Character',
           'CharacterNormalMap',
           'CharacterSpecular',
           'Weapon',
           'WeaponNormalMap',
           'WeaponSpecular',
           'Vehicle',
           'VehicleNormalMap',
           'VehicleSpecular',
           'Cinematic',
           'Effects',
           'EffectsNotFiltered',
           'Skybox',
           'UI',
           'Lightmap',
           'RenderTarget',
           'MobileFlattened',
           'Shadowmap',
           'IESLightProfile'
        ]

#define a texture class to perform the order script later
class Texture(object):
    def __init__(self, size, name):
        self.size = size
        self.name = name
        memory = size.split('x')
        self.memorySpace = int(memory[0])

def orderTextures(filePath, outPath, filterChoose = None):
    #opens the file and iterate the csv, filtering the rows if asked by user checking the filterChoose var
    #the try is for handling the exception caused by the user closing the windows during file selection
    try:
        with open(filePath,'rb') as csvfile:
            textureList = list(csv.reader(csvfile, delimiter=';'))
            del textureList [0]
            textureObjects=[]

            if filterChoose:
                textureList = filterByGroup(textureList, filterChoose)

            #the streaming textures info extracted with the console command in UE4 didn't change in 15 versions,
            #but if they change the rows and columns info you'll want to re-check this piece of code
            for row in textureList:
                textureObject = Texture(row[2], row[7])
                textureObjects.append(textureObject)

            #this is the sorting part, I use the memorySpace attr of the texture object
            sortedTextures = sorted(textureObjects, key= lambda texture: texture.memorySpace, reverse = True)

            with open(outPath,'w') as outFile:
                for textureObject in sortedTextures:
                    outFile.write("Name = " + textureObject.name + " | Size = " + textureObject.size + "\n")
    except:
        exit
    finally:
        showinfo ('Streaming textures analyzer', 'Streaming textures report saved in %s' %outPath)

def filterByGroup(textureList, textureGroup):
    filteredTextures = []

    #the streaming textures info extracted with the console command in UE4 didn't change in 15 versions,
    #but if they change the rows and columns info you'll want to re-check this piece of code
    for row in textureList:
        if textureGroup not in row[6]:
            continue
        else:
            filteredTextures.append(row)

    return filteredTextures

def orderingCallback(filterCheckValue, filterChoose):
    if not filterCheckValue.get():
        filterChoose = None
    else:
        filterChoose = str(filterChoose.get())
    tk.Tk().withdraw()

    showinfo ('Streaming textures analyzer', 'Select the CSV streaming texture file')
    filePath = askopenfilename()

    #asksave file returns a file in write mode, I used name to return just the path string to use it in the orderTextures function
    showinfo ('Streaming textures analyzer', 'Select the path where to save the report file')
    outPath = (asksaveasfile(mode = 'w', defaultextension ='.txt')).name

    orderTextures(filePath, outPath, filterChoose)

#this is the UI logic
startWindow = tk.Tk()
startWindow.geometry("%dx%d+%d+%d" % (350, 150, 200, 150))
startWindow.title("Texture Streaming CSV Analyzer")

filterCheckValue = IntVar()
filterChoose = tk.StringVar(startWindow)

tk.Button(startWindow, text = 'Analyze texture streaming file', command = lambda: orderingCallback(filterCheckValue, filterChoose)).pack(fill = 'x')
tk.Checkbutton (startWindow, text = 'Filter texture by group', variable = filterCheckValue, onvalue = 1, offvalue = 0, height= 5, width = 20).pack()

filterChoose.set('World')

filterOptions = tk.OptionMenu(startWindow, filterChoose, *textureFilterChoices)
filterOptions.pack(side='bottom', padx=45, pady=10)

tk.mainloop()

I am sorry that I can’t provide the c++ class to edit for exporting the CSV, by the way I really hope this script will help someone as it did for me!
I wrote some comments in the code and the use should be really intuitive, but for every doubts please write me and I’ll answer you as soon as I can.

Bye!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s