grow your own tools - vilniusrb
TRANSCRIPT
![Page 1: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/1.jpg)
Grow your own toolsdebugging & profiling story
Remigijus Jodelis - SameSystem
VilniusRB Vilnius Ruby Community
![Page 2: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/2.jpg)
Part I
Debugging
![Page 3: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/3.jpg)
binding.pry
Debugging
![Page 4: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/4.jpg)
Debugging
puts
![Page 5: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/5.jpg)
Debugging
Rails.logger.info
![Page 6: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/6.jpg)
Debugging
p some, thing
![Page 7: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/7.jpg)
Debugging
pp some, thing
![Page 8: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/8.jpg)
Debugging#<ActiveSupport::Logger:0x007f1aa39426f8 @default_formatter=#<Logger::Formatter:0x007f1aa3942450 @datetime_format=nil>, @formatter= #<ActiveSupport::Logger::SimpleFormatter:0x007f1aa4c0c900 @datetime_format=nil>, @level=0, @logdev= #<Logger::LogDevice:0x007f1aa39422c0 @dev=#<File:/vagrant/samesystem/log/development.log>, @filename=nil, @mutex= #<Logger::LogDevice::LogDeviceMutex:0x007f1aa3942298 @mon_count=0, @mon_mutex=#<Mutex:0x007f1aa3942090>, @mon_owner=nil>, @shift_age=nil, @shift_size=nil>, @progname=nil>=> #<ActiveSupport::Logger:0x007f1aa39426f8 @progname=nil, @level=0, @default_formatter=#<Logger::Formatter:0x007f1aa3942450 @datetime_format=nil>, @formatter=#<ActiveSupport::Logger::SimpleFormatter:0x007f1aa4c0c900 @datetime_format=nil>, @logdev=#<Logger::LogDevice:0x007f1aa39422c0 @shift_size=nil, @shift_age=nil, @filename=nil, @dev=#<File:/vagrant/samesystem/log/development.log>, @mutex=#<Logger::LogDevice::LogDeviceMutex:0x007f1aa3942298 @mon_owner=nil, @mon_count=0, @mon_mutex=#<Mutex:0x007f1aa3942090>>>>
SHIT...
![Page 9: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/9.jpg)
Debugging
We need to do better
![Page 10: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/10.jpg)
Debugging
STEP 1
Better name
![Page 11: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/11.jpg)
Debugging
# initializers/wtf.rb
def WTF?(*args) p argsend
![Page 12: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/12.jpg)
Debugging def sync_records(resource, local_idx, remote_idx, key_name, options={}) options.reverse_merge!(@opt)
cross_idx, actions = diff_collection(local_idx, remote_idx) actions.each do |action, items| @cnt["#{resource.name}_#{action}"] = items.size end WTF? resource.name.to_s, *actions, :line, :file
actions.each { |action, items| items.clear unless options[action] }
if options[:create_many] resource.create_many(*actions[:create]) if actions[:create].any? else actions[:create].each do |record| remote = resource.create(record) or next if local = local_idx[remote[key_name.to_s]] cross_idx[local.id] = remote['id'] end
![Page 13: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/13.jpg)
Debugging
STEP 2
More options
![Page 14: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/14.jpg)
Debugging
def WTF?(*args) if args.last == :pp args.pop pp args else p args endend
WTF? my_struct, :pp
![Page 15: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/15.jpg)
Debugging
WTF? my_struct, :pp
WTF? my_var, :yamlWTF? my_var, other_var, :pp, :fileWTF? *other_things, :json, :file, :time
But I want even more...
![Page 16: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/16.jpg)
Debugging
def WTF?(*args) allowed = [:pp, :yaml, :json] options = {} while allowed.include?(args.last) options[args.pop] = true end
case when options[:yaml] puts args.to_yaml # ...
![Page 17: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/17.jpg)
Debugging
data = "" data << "[%s] " % Time.now if options[:time] rx = %r{([^/]+?)(?:\.rb)?:(\d+):in `(.*)'$} # <- WTF is this? data << "WTF (%s/%s:%s)" % caller[0].match(rx).values_at(1,3,2)
data << ": " << case when options[:pp] args.pretty_inspect.gsub(/^\[|\]$/,'') # removes array marks when options[:yaml] YAML.dump(args) when options[:json] JSON::pretty_generate(args) when options[:text] args.map(&:to_s).join("\n")
![Page 18: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/18.jpg)
Debugging
WTF (search_controller/browse:15): #<Paginator:0x007fd0199e6bf0 @count=169, @per_page=20,
def browse if params[:search].present? joins = search_joins conditions = search_conditions includes = search_includes count = klass.joins(joins).includes(includes).where(conditions).count pager = ::Paginator.new(count, search_config.per_page) do |offset, per_page| klass.joins(joins).where(conditions).includes(includes). endWTF? pager, :pp
![Page 19: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/19.jpg)
Debugging
STEP 3
Moreoutput options
![Page 20: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/20.jpg)
Debugging
case when options[:page] (Thread.current[:wtf] ||= []) << data when options[:file] time = Time.now.strftime('%m%d_%H%M%S') filename = "wtf_#{time}_#{rand(10000)}.txt" File.write("#{Rails.root}/tmp/#{filename}", data) when options[:raise] raise data when options[:redis] REDIS.rpush 'wtf', data REDIS.expire 'wtf', 30*60 else Rails.logger.info data end
![Page 21: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/21.jpg)
Debugging
OK, I got it
But, where do we place all this code?
![Page 22: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/22.jpg)
Debugging
module Kernel WTF_OPTIONS = [:time, # prefix :pp, :yaml, :json, :csv, :text, :line, # format :bare, # modifiers :page, :file, :raise, :redis, :log] # output
def WTF?(*args)
The same place as p
![Page 23: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/23.jpg)
Debugging
Time to refactor!
![Page 24: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/24.jpg)
Debugging
Object.class_eval do def WTF?(*args) WTF::Dumper.new(*args) endend
module WTF class Dumper OPTIONS = [:time, :nl, :none, # ...
![Page 25: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/25.jpg)
Part II
Profiling
![Page 26: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/26.jpg)
So we got a situation...
![Page 27: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/27.jpg)
Profiling
class Overview # loading dependent data and calculations def load! load_departments
@budget_repo = Overviews::BudgetRepository.new(@period, @shops) @data = {}
columns.each do |col| calculate(col) end # … few more lines … end
# … 50 more methods …
The problem lurks somewhere here
![Page 28: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/28.jpg)
Profiling
Let’s try the profiler
![Page 29: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/29.jpg)
Profiling
![Page 30: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/30.jpg)
Profiling
WTF am I looking at?
![Page 31: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/31.jpg)
Profiling
Second shot
start with a simple thing
![Page 32: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/32.jpg)
Profiling
# loading dependent data and calculations def load! st = Time.now
# ...
WTF? Time.now - st end
WTF (overview/load!:250): 180.3073...
![Page 33: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/33.jpg)
Profiling
What if I tried this
on EVERY methodin the same class
![Page 34: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/34.jpg)
Profiling
module MethodTracker class << self def included(base) methods = base.instance_methods(false) + base.private_instance_methods(false) base.class_eval do methods.each do |name| original_method = instance_method(name) define_method(name) do |*args, &block| MethodTracker.on_start(base, name) return_value = original_method.bind(self).call(*args, &block) MethodTracker.on_end return_value end end end end
![Page 35: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/35.jpg)
Profiling
In Ruby 2.x we can do nicer
hint: prepend OverridesModule
![Page 36: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/36.jpg)
Profiling
def override_method(base, name) %{ def #{name}(*args) WTF::MethodTracker.on_start(#{base}, :#{name}) return_value = super WTF::MethodTracker.on_end return_value end } end
![Page 37: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/37.jpg)
Profiling
def prepare(base) methods = base.instance_methods(false) + base.private_instance_methods(false) compiled = methods.map do |name| override_method(base, name) end base.module_eval %{ module Tracking #{compiled.join} end prepend Tracking } end
![Page 38: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/38.jpg)
Public interfacemodule WTF class << self def track(*objects) MethodTracker.setup(*objects) MethodTracker.reset_state end
def track_finish MethodTracker.finish end endend
class Overview def load! WTF.track(self, Overview::Helpers) # with additional classes # ... WTF.track_finish endend
![Page 39: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/39.jpg)
Collecting data
require 'absolute_time'
def add_stats(at_start = nil) stat = stats[stack.last]
this_time = AbsoluteTime.now stat[:time] += this_time - last_time self.last_time = this_time
this_heap = GC.stat[:heap_length] stat[:heap] += this_heap - last_heap self.last_heap = this_heap
stats[at_start][:freq] += 1 if at_start end
![Page 40: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/40.jpg)
Profiling
What about the output?
![Page 41: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/41.jpg)
Profiling
![Page 42: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/42.jpg)
Profiling
![Page 43: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/43.jpg)
Profiling
Let’s look at one more problem.
I got some Rails logs...
![Page 44: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/44.jpg)
Profiling
WTF is generating this query?
![Page 45: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/45.jpg)
Profiling
I want to do this
# loading dependent data and calculationsdef load!
WTF.sql %(SELECT `weekly_balance_lines`.* FROM `weekly_balance_lines
load_shops
@budget_repo = Overviews::BudgetRepository.new(@period, @shops) @salary_repo = Overviews::SalaryRepository.new(@period, @shops, @c
![Page 46: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/46.jpg)
Profiling
… and get the answer
SQL: "SELECT `weekly_balance_lines`.`date`, `weekly_balance_lines`.`context_id`, "(eval):4:in `log'" "/home/vagrant/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/activerecord-4.1 "/home/vagrant/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/activerecord-4.1 "/home/vagrant/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/activerecord-4.1 "/home/vagrant/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/activerecord-4.1 "/home/vagrant/.rbenv/versions/2.1.5/lib/ruby/gems/2.1.0/gems/activerecord-4.1 "/vagrant/samesystem/app/models/overview.rb:350:in `weekly_balance_lines_for_p "/vagrant/samesystem/app/models/overview.rb:342:in `weekly_balance_lines'" "/vagrant/samesystem/app/models/overview.rb:870:in `sales'" "/vagrant/samesystem/app/models/overview.rb:396:in `calculate'" "/vagrant/samesystem/app/models/overview.rb:241:in `block in
![Page 47: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/47.jpg)
Profiling
ActiveRecord::ConnectionAdapters::AbstractAdapter.module_eval %( module TrackingSQL def log(sql, *args, &block) WTF::QueryTracker.on_sql(sql) # callback and WTF? caller super(sql, *args, &block) end end prepend TrackingSQL)
Easy! The key piece is here
![Page 48: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/48.jpg)
Conclusion
Does it look like a gem?
YES!
![Page 49: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/49.jpg)
github.com/remigijusj/wtf-tools
Please check out the repo!
![Page 50: Grow your own tools - VilniusRB](https://reader034.vdocuments.site/reader034/viewer/2022051520/58f1c6741a28abf9318b4601/html5/thumbnails/50.jpg)
Thank you!