go & multi platform gui trials and errors
TRANSCRIPT
Go & multi platform GUITrials and Errors
Go Conference 2016 sprint @ 4/23/2016@shibukawa (Twitter)github.com/shibukawa
Yoshiki Shibukawa
Who Are You?• Programmer in Mobile Game Department
– http://www.slideshare.net/dena_study/final-fantasy-record-keeper
Yoshiki Shibukawa
Languages C++/JavaScript/Python/Golang etc Japanese/English/Español (studying with Duolingo)
Books Translations: Expert Python/Pomodoro Technique/The Art of
community Original: Mithril, つまみぐい勉強 etc
Before talking my presentation…
• Machine readable docs for web service API
Before talking my presentation…
• Machine readable docs for web service API
YES!
• https://shibukawa.github.io/curl_as_dsl/• I will add net/http, fasthttp server code generate in future• Welcome Ruby/Perl/emacs lisp pull request
Conclusion
• This presentation doesn’t show the best solution for multi platform GUI
• I am exploring possibility about Go and GUI and creating experimental codes.
• GUI programming is not easy• C++ is required
Solutions I triedPros Cons
Toolkit Wrapper The most intuitive solution Maintaining wrappers is extremely hard jobCan’t use deploy support tool
Create new Toolkit in Go Perfect matching with golang
Supporting multi platform is very tough
Hybrid (C++ for GUI, Go for logic)
Solution 1Toolkit Wrapper
Toolkit Wrapper
• I used wxPython, Qt for JavaScript etc…• The most intuitive solution
– You can use it in same methodology with original native language
I tried go-qml
http://qiita.com/shibukawa/items/47cb0ea60bf4a7259b5e
Toolkit Wrapper Cons (1)
• Qt provides deployment helper tool– macdeployqt, windeployqt etc– They collects required .dll/.framework and rewrite
dependency paths and create installer etc– This tool works only for Pre build Qt/C++ project
• Few documents/examples– We reinterpret original toolkit documents– C++ knowledge is required
• Debug is very hard– Where does this panic come from?
Toolkit Wrapper Cons (2)
• Maintaining toolkit wrapper is hard work– GUI toolkit is contains many C++ classes
• Qt has more than 1200 classes– Many wrappers were discarded
• QtJS, QtJambi, ruby qtbindings, go-qml etc…– GUI toolkits update continuously
• Basically newer version is better• Almost all toolkit gave up updating• I have to update wrapper by myself to use newer future
– Languages update too
Expect
Hours
Create My ApplicationMaintaining Toolkit
Actually
Hours
Create My ApplicationMaintaining Toolkit
Not all GUI toolkit wrappers are bad
PyQt Commercial (Qt 5.5 supports)
wxPython Nightly build still supports latest Python(2.7/3.5)
GTK# Xamarin supports?
pygobject or some libs gobject introspection is good for keeping fresh
Swing from Jython/JRuby Using toolkit directly
tkinter / Qt Quick Official wrapper provided by suppliers
• Some wrappers provide updates continuously• Maintaining is too heavy for individual developer• Community/Organizational support is required
Solution 2Create New Toolkit in Go
Implement GUI toolkit in Go• There are many small GUI toolkits in C/C++
– http://qiita.com/shimacpyon/items/e5d2a4e2019273345c37• Port one of them to Go!
– NanoGUI seems good to port
※ When start porting, I didn’t the existence of Shiny
GUI toolkit on GL building blocks
Font Loader Font Renderer
Texture Management
Font TextureManagement
Event Handling
Vector GraphicsImage Loader
Widget Drawing
OpenGLOS Window System
Widget Management
NanoGUI toolkit on GL building blocks
Font Loader Font Renderer
Texture Management
Font TextureManagement
Event Handling
Vector GraphicsImage Loader
Widget Drawing
OpenGLOS Window System
Widget Management
stb_truetype.h
stb_image.h
NanoVG
GLFW
NanoGUI
fontstash
My toolkit on GL building blocks
Font Loader Font Renderer
Texture Management
Font TextureManagement
Event Handling
Vector GraphicsImage Loader
Widget Drawing
OpenGLOS Window System
Widget Management
go’s image
NanoVGo
goxjs/glfwgoxjs/gl
NanoGUI.go
nanovgo
nanogui-go
Input Method Editor issue
• IME support is needed to input Asian chars• Now IME becomes more generic than before
– Voice / pen input uses same infrastructure– Modern OS treats IMEs as keyboards
• This is processed in GLFW (C++ layer)Japanese IME
English
Speech Input
Handwriting Input
OperatingSystem
Application
※On Windows, Handwriting features seem to be a part of MS IMEs
IME support levels
• Level 1: Root window mode
• Level 2: Over the spot
Images from http://www.mozilla-japan.org/projects/intl/input-method-spec.htmlThis is not common classify. My original.※
• Level 3: On the spot
• Level 4: Context aware conversion
Image from http://nyaruru.hatenablog.com/entry/20070309/p1
IME tasks viewed from ApplicationLevel 1 Level 2 Level 3 Level 4
Receive multi byte input ✔ ✔ ✔ ✔
Send cursor position ✔ ✔ ✔
React to preedit text ✔ ✔
Send surrounding text around cursor
✔
• To achieve higher level IME support, applications should communicate with IME more
• Each OS provides each API for IME. I should add new method to GLFW to absorb OS layers
Achievement Unlocked!
Current Status
• Windows– Level 3. Now I use IMM32(old), but Text Services
Framework (TSF) is preferred.– Old Chrome and Firefox use TSF (but MPL is not
good for GLFW)• MacOS X
– Level 3. Applications should implement NSTextInputClient protocol. I don’t implement Level 4 feature yet.
https://github.com/glfw/glfw/pull/658
Current Status
• X11– Level 3. XIM protocol doesn’t support level 4.– On Linux/BSDs, there many input method frameworks
(UIM, SCIM, ibus, fcitx etc) and Qt/GTK communicate with them directly, but don’t use them (not to introduce new dependency to GLFW)
• Wayland– Not yet
• Mir– Not yet
https://github.com/glfw/glfw/pull/658
Future Plan
• Improve GLFW more to support level 4 and Wayland, Mir
• Reconstruct NanoVGo– Cache geometry – Now it redraw from scratch on every frame
• Reconstruct NanoGUI.go– ListView support like QAbstractItemView /
QAbstractItemModel– Better image resource handling
Solution 3Hybrid Style
Hybrid Model
Thin GUI Logic
C++ is not so fear than before
• Use Qt via C++ directly– No wrapper issues– Using C++ is not tough anymore– C++11 is much better than 20 years ago– Qt5 supports C++11 (C++11 is required from 5.7)
• But building code in Golang is still faster than C++ for me
C++ support tool• Go is easy to build. C++ is boring• I made helper tool to make C++ easy
– https://github.com/qtpm/qtpm• Generate CMakeLists.txt. Support following IDEs:
– Qt Creator– CLion (thank you for Jetbrains OSS support program)
• It support the following commands like go:– qtpm get– qtpm build– qtpm vet– qtpm fmt
How to communicate?
Thin GUI Logic
???
• I am trying several solutions for communication
Qt’s thread model
• You should understand it if use Go with Qt• http://doc.qt.io/qt-5/threads-qobject.html• All Qt’s objects are belonging to thread
– Calling object’s method from non-owner thread causes panic
– To call other thread’s method, QMetaObject::invokeMethod is needed(signal/slot uses this behind it)
– Each thread has event loop– Of course, blocking main thread is not preferred
3.1: QProcess
• Create standalone CUI program in Golang and call via QProcess class– If the job is simple, it is the best (it becomes
simple code)• Bi-directional communication is tough
– Receiving message in SLOT (callback)– No fluent code
3.2: QLocalServer/QLocalSocket
• Qt provides special wrapper to provides high performance inter process communication– On Windows, it uses named pipe– On other environment, it uses UNIX domain socket
• Go implementation is ready– I tested on Windows and Mac
• How to manage process?– I need to implement some process supervisor
https://github.com/shibukawa/localsocket
3.3: c-archive
• Creating c-archive and link with Qt application• Go 1.7 will support it on Windows too
• If the API is simple, it is the best• All method is called synchronous by default• If callback from Golang to C++, it is little messy
– Polling is easy, but not beautiful– Pass function pointer from C++ and call it in Go
• https://gist.github.com/shibukawa/ab516964b0390cb234c7c42e5c9bfda1
• How to specify “-mmacosx-version-min=10.x”?
3.4: dRuby style
• dRuby is a my favorite API in Ruby• dRuby hooks method call and transfers
method name and parameters via TCP/IP– Go: https://github.com/shibukawa/tobubus– C++: https://github.com/qtpm/QtTobubus
DRbObjectClient
hello(“world”)
method_missing()
DRbServerActualInstance
hello(“world”)
tobubus
• Now it works on QLocalSocket/Server• It uses Qt
reflection(QMetaObject/QMetaMethod) and Go’s reflect
• Method call is serialized by CBOR– CBOR is almost RFC version of MessagePack
tobubusC++/Go Client
hello(“world”)
ProxyC++/Go ActualInstance
hello(“world”)
tobubus
CBOR on local socket
3.5: gRPC
• I didn’t try it– I don’t want to create extra interface definitions
for communication• It is suitable for loose coupling of micro services
– It seems difficult to use with Qt threading model• I may need create wrappers for all interfaces in Qt
Combinations of my solutionsInfrastructure
Connect Pros/Cons Simple Tested
QProcess stdin/stdout
bi-direction is difficultGo is always running in other process (should take care QThread)
S ✔
tobubus Not implemented yet A
QProcess + QLocalSocket/Server
tobubus It needs process supervisorC++ client code is not clean because of Qt thread modelGo is always running in other process (should take care QThread)
B ✔
c-archive linker Writing callback is little messyWindows doesn’t support until 1.7It works synchronously
S ✔
Which is better?
• QProcess is the most simple now• tobubus + QLocalSocket/Server was originally
designed for generic plug-in system. I already implemented but I feel too heavy now.
• c-archive is a stable way. It is not bad if you can wait by 1.7 release.
Conclusion
• This presentation doesn’t show the best solution for multi platform GUI
• I am exploring possibility about Go and GUI and creating experimental codes. – I will still improve GLFW IME support to realize pure
Go solution– If you can wait by Go 1.7, c-archive is better
• GUI programming is not easy• C++ is required
Actually
Hours
Create My ApplicationMaintaining Toolkit
In future I want to achieve
Hours
Create My ApplicationMaintaining Toolkit