first do no harm: surgical refactoring

First Do No HarmSurgical Refactoring

Nell Shamrell-Harrington@nellshamrell

Section I: Refactoring

First Do No Harm: Surgical Refactoring @nellshamrell


Refactoring is a change and changes can go wrong

What is refactoring?

But…it’s not life or death, right?




Software is being integrated into…




Transportation




TransportationEnergy sources




TransportationEnergy sourcesMedical Devices

So…is refactoring bad, then?




Refactoring is neither inherently good OR bad



How you do it is what matters

How should I refactor?


2 Common Approaches



2 Common Approaches1) Edit and Pray




2) Cover and Modify




2) Cover and Modify- “Working Effectively with Legacy Code”

Section II: Surgical Refactoring


What is surgical refactoring?


Change exactly what we INTEND to changeAnd ONLY what we intend to change



First do no harm!


Surgical refactoring is a series of good habits that reduce risk



2 categories of refactoring





1) Necessary refactoring




1) Necessary refactoring2) Cosmetic refactoring

What is a necessary refactoring?


Need to add something



Need to add somethingCode is too inefficient



Need to add somethingCode is too inefficient

Blocked from achieving a business need



Necessary refactorings have a moderate to high risk tolerance

What is a cosmetic refactoring?




No business need to change code



Something about it just bugs usNo business need to change code



Cosmetic refactorings have a low risk tolerance

What about whitespace refactoring?




Ultimate cosmetic refactoring




Get a style guide (i.e. Github style guide)




Get a style guide (i.e. Github style guide)

If whitespace does not violate style guide, leave it alone!

What’s involved in surgical refactoring?




1) Pre-op: what to do before touching the code



1) Pre-op: what to do before touching the code 2) Operation: doing the actual refactoring



1) Pre-op: what to do before touching the code 2) Operation: doing the actual refactoring

3) Recovery: verifying the refactor


lib/do_system_things.rb

class DoSystemThings

def do_the_thing(dir) system "sed -E -i '' 's/:([[:alnum:]]+)[[:space:]]=>/\\1:/g' #{dir}/**/*.rb" end

end

Thanks, @mikelorant!

Section IV: Pre-Op


What is involved in pre-op?


Diagnosis (What exactly does the code do?)





end





end Calls Ruby’s system method





endExecutes sed command with some flags





end

Performs a substitution





endIn a series directories and files

So…do we know what it does?




Map is influenced by our own experiences and expectations



Only definite way of knowing what the code does is to execute the code itself



Best way to repeatedly execute the code is through automated tests





end

Now what does this system call do?


spec/do_system_things_spec.rb

let(:do_system_things) { DoSystemThings.new }

Instantiate the class



let(:do_system_things) { DoSystemThings.new } let(:dir) { ‘something’ }

Sample argument to pass to class




describe ‘making the system call’ do it ‘calls the Ruby#system method’ do

end

end




describe ‘making the system call’ do it ‘calls the Ruby#system method’ do expect(do_system_things)

.to receive(:system).with(anything())

end end

Expect that our instance of the class



Will receive a system call with any args



.to receive(:system).with(anything())

end end





.to receive(:system).with(anything()) do_system_things.do_the_thing(dir)

end end Call the method





.to receive(:system).with(anything()) do_system_things.do_the_thing(dir)

end end

Spec Passes!




def do_the_thing(dir) "sed -E -i '' 's/:([[:alnum:]]+)[[:space:]]=>/\\1:/g' #{dir}/**/*.rb" end

end

Remove the system call




def do_the_thing(dir) "sed -E -i '' 's/:([[:alnum:]]+)[[:space:]]=>/\\1:/g' #{dir}/**/*.rb" end

end

Spec Fails!





end

Put the system call back





end

Spec Passes!





endWhat return is expected?





end Rdocs: system call returns true when the command executes successfully



let(:dir) { ‘something’ }

context ‘when the method call is successful’ do it ‘returns true’ do end end




context ‘when the method call is successful’ do it ‘returns true’ do expect(do_system_things.do_the_thing(dir)).to eq(true)

end end

Expect the return from calling the method on the instance of the class





end end

To return true





end end

Spec Fails!



'sed: directory/*/.rb: No such file or directory'



end end



context ‘when the method call is successful’ do let(:directory_name) { ‘some_directory’ } let(:sub_directory) { ‘some_sub_directory’ } let(:file) { ‘some_file.rb’ }

it ‘returns true’ do expect(do_system_things.do_the_thing(dir)).to eq(true)

end end

Set the directory, subdirectory, and file names




before do FileUtils.mkdir_p(File.join(directory_name, sub_directory)) end


end end

Create directories and sub-directories




before do FileUtils.mkdir_p(File.join(directory_name, sub_directory)) path = File .join(directory_name,subdirectory_name,file_name) end


end end

Create the path for the file




before do FileUtils.mkdir_p(File.join(directory_name, sub_directory)) path = File .join(directory_name,subdirectory_name,file_name) file = File.new(path, ‘w’) file.write(‘look, there is something in this file’) file.close end

it ‘returns true’ do

Create the actual file






Spec Passes!






That’s a lot of setup code…



def create_required_directories_and_files(path, filename) FileUtils.mkdir_p(File.join(path)) file = File.new(File.join(path,file_name), ‘w’) file.write(‘look, there is something in this file’) file.close end Takes a path and

file name



def create_required_directories_and_files(path, filename) FileUtils.mkdir_p(File.join(path)) file = File.new(File.join(path,file_name), ‘w’) file.write(‘look, there is something in this file’) file.close end Makes the directories



def create_required_directories_and_files(path, filename) FileUtils.mkdir_p(File.join(path)) file = File.new(File.join(path,file_name), ‘w’) file.write(‘look, there is something in this file’) file.close end

Makes the file



before do create_required_directories_and_files( File.join(directory_name,subdirectory_name), file_name ) end


end

Call the setup method





end

Create and pass the path for the file





end Pass the file name





end

Spec Passes!




it ‘returns true’ do expect(do_system_things.do_the_thing(dir)).to eq(true) end

after do FileUtils.rm_rf(directory_name) end Remove created

directories and file




it ‘returns true’ do expect(do_system_things.do_the_thing(dir)).to eq(true) end

after do FileUtils.rm_rf(directory_name) end

Spec Passes!





end

*pause*





end

sed is a streaming text editor





end



Find match for this pattern

s/:([[:alnum:]]+)[[:space:]]=>/\\1:/g'



Replace it with this pattern

s/:([[:alnum:]]+)[[:space:]]=>/\\1:/g'




def first_regex_match(string)

end


end




def first_regex_match(string) //.match(string) end


end



describe ‘what the regex matches’ # FOR REFERENCE: /:([[:alnum:]]+)[[:space:]]=>/

end




let(:string) {‘'}

end




let(:string) {''} it 'matches a string' do

end end




let(:string) {''} it 'matches a string' do expect(do_system_things.first_regex_match(string)) .not_to be_nil end end




def first_regex_match(string) //.match(string) end


end

Returns nil if no successful match





Means it found a successful match





first character is ‘:’




let(:string) {‘:'} it 'matches a string' do expect(do_system_things.first_regex_match(string)) .not_to be_nil end end add ‘:’ to string




let(:string) {‘:'} it 'matches a string' do expect(do_system_things.first_regex_match(string)) .not_to be_nil end end

Spec Fails!




def first_regex_match(string) /:/.match(string) end


end

add ‘:’ to regex




def first_regex_match(string) /:/.match(string) end


end

Spec Passes!




let(:string) {‘:'} it 'matches a string' do expect(do_system_things.first_regex_match(string)) .not_to be_nil end end

Next character is alnum character class




let(:string) {‘:a'} it 'matches a string' do expect(do_system_things.first_regex_match(string)) .not_to be_nil end end Add letter to test string



Spec Fails!


let(:string) {‘:a'} it 'matches a string' do expect(do_system_things.first_regex_match(string)) .not_to be_nil end end




def first_regex_match(string) /:[[:alnum:]]/.match(string) end


end

add alnum to regex






end

Spec Passes!




let(:string) {‘:a'} it 'matches a string' do expect(do_system_things.first_regex_match(string)) .not_to be_nil end end Means character must

appear one or more times




let(:string) {‘:ab'} it 'matches a string' do expect(do_system_things.first_regex_match(string)) .not_to be_nil end end

Add character to test string



Spec Passes!





But…we were expecting a failure…








end Will return successful match with only one character




let(:string) {‘:ab'} it 'matches a string' do expect(do_system_things.first_regex_match(string) .to_s).to include(‘:ab’) end

end

Examining content of string that was captured by the regex



Spec Fails!



end




def first_regex_match(string) /:[[:alnum:]]+/.match(string) end


end

add + quantifier




def first_regex_match(string) /:[[:alnum:]]+/.match(string) end


end

Spec Passes!



# FOR REFERENCE: /:([[:alnum:]]+)[[:space:]]=>/


Capture group





it ‘captures a group’ do

end





it ‘captures a group’ do expect(do_system_things.first_regex_match(string)[1]) .not_to be_nil end Testing that match captures

a capture group ([1] references first capture group)



Spec Fails!



it ‘captures a group’ do expect(do_system_things.first_regex_match(string)[1]) .not_to be_nil end




def first_regex_match(string) /:[[:alnum:]]+()/.match(string) end


endAdding empty capture group




def first_regex_match(string) /:[[:alnum:]]+()/.match(string) end


end

Spec Passes!





it ‘captures a group’ do expect(do_system_things.first_regex_match(string)[1]) .not_to be_nil expect(do_system_things.first_regex_match(string)[1]) .to eq(‘ab’) end

Testing content of capture group



Spec Fails!



it ‘captures a group’ do expect(do_system_things.first_regex_match(string)[1]) .not_to be_nil expect(do_system_things.first_regex_match(string)[1]) .to eq(‘ab’) end




def first_regex_match(string) /:([[:alnum:]]+)/.match(string) end


endPlacing capture group in correct place




def first_regex_match(string) /:([[:alnum:]]+)/.match(string) end


end

Spec Passes!




def first_regex_match(string) /:([[:alnum:]]+)[[:space]]=>/.match(string) end


end Fast Forward…




def first_regex_match(string) /:([[:alnum:]]+)[[:space]]=>/.match(string) end


end We know what this regex does!


spec/do_system_things_spec.rbcontext ‘changing the file’ do before do file = File.open(File.join(directory_path,file_name)) file.write(':ab =>') file.close end end

Write sample string to file


spec/do_system_things_spec.rbcontext ‘changing the file’ do it ‘changes the file’ do original_contents = File.read(File.join(directory_path,file_name))

end end

Capture original contents of file



do_system_things.do_the_thing(directory_name)

end end Call the method




new_contents = File.read (File.join(directory_path,file_name)) end end

Capture new contents of file




new_contents = File.read (File.join(directory_path,file_name)) expect(original_contents).not_to eq(new_contents) end end

Make sure file changes




new_contents = File.read (File.join(directory_path,file_name)) expect(original_contents).not_to eq(new_contents) end end

Spec Passes!





end

*pause*



Replace match for the first pattern with this second pattern

s/:([[:alnum:]]+)[[:space:]]=>/\\1:/g'



\\1:



\\1:

Escape - so we can use a literal \ as the next character



\\1:

Uses the result of the first capture group from the first pattern



\\1:

Adds a literal colon



s/:([[:alnum:]]+)[[:space:]]=>/\\1:/g'

‘:ab =>’



s/:([[:alnum:]]+)[[:space:]]=>/\\1:/g'

‘ab:’‘:ab =>’ replace with


spec/do_system_things_spec.rbcontext ‘changing the file’ do let(:orig_string) { “:ab => “} let(:new_string) { “ab: ‘}

it ‘changes the file’ do original_contents = File.read(File.join(directory_path,file_name))


new_contents = File.read(File.join(directory_path,file_name)) end end

Expected content of file before and after





new_contents = File.read(File.join(directory_path,file_name)) expect(new_contents).not_to include(orig_string) expect(new_contents).to include(new_string) end end

Verify contents of modified file





new_contents = File.read(File.join(directory_path,file_name)) expect(new_contents).not_to include(orig_string) expect(new_contents).to include(new_string) end end Spec Passes!





end

*pause*



s/:([[:alnum:]]+)[[:space:]]=>/\\1:/g'

Replace all matches for the first pattern with the second pattern


spec/do_system_things_spec.rbcontext ‘changing the file’ do let(:orig_string) { “:ab => :ab =>“} let(:new_string) { “ab: ab:’}

it ‘changes all matches within the file’ do original_contents = File.read(File.join(directory_path,file_name))


new_contents = File.read(File.join(directory_path,file_name)) expect(new_contents).not_to include(orig_string) expect(new_contents).to include(new_string) end end

Expected content of file before and after


spec/do_system_things_spec.rbcontext ‘changing the file’ do let(:orig_string) { “:ab => :ab =>“} let(:new_string) { “ab: ab:’}

it ‘changes all matches within the file’ do original_contents = File.read(File.join(directory_path,file_name))

do_system_things(directory_name)

new_contents = File.read(File.join(directory_path,file_name)) expect(new_contents).not_to include(orig_string) expect(new_contents).to include(new_string) end end Spec Passes!




def do_the_thing(dir) system "sed -E -i '' ’s/:([[:alnum:]]+)[[:space:]]=>/\\1:/" #{dir}/**/*.rb" end

end

Taking out the global flag




def do_the_thing(dir) system "sed -E -i '' ’s/:([[:alnum:]]+)[[:space:]]=>/\\1:/" #{dir}/**/*.rb" end

end

Spec Fails!




def do_the_thing(dir) system "sed -E -i '' ’s/:([[:alnum:]]+)[[:space:]]=>/\\1:/g" #{dir}/**/*.rb" end

end

Putting the flag back in





end

Spec Passes!




def do_the_thing(dir) system "sed -i '' ’s/:([[:alnum:]]+)[[:space:]]=>/\\1:/g" #{dir}/**/*.rb" end

end

Take -E flag out




def do_the_thing(dir) system "sed -i '' ’s/:([[:alnum:]]+)[[:space:]]=>/\\1:/g" #{dir}/**/*.rb" end

end

Spec Fails!





end

Put -E flag back in





end

Spec Passes!





end

-i flag: sed alters files in place





end

-i flag: sed alters files in place

Passing an empty extension


spec/do_system_things_spec.rbcontext ‘changing the file’ do

it ‘does not save a backup copy of the file’ do expect (Dir[“#{directory_name}/#{subdirectory_name}/*”] .count).to eq(1)

end end

Expect only one file in directory/subdirectory path





end end

Call the method





expect( Dir[“#{directory_name}/#{subdirectory_name}/*”] .count).to eq(1) end end

Expect only one file in directory/subdirectory path





expect (Dir[“#{directory_name}/#{subdirectory_name}/*”] .count).to eq(1) end end

Spec Passes!




def do_the_thing(dir) system "sed -E -i .tmp ’s/:([[:alnum:]]+)[[:space:]]=>/\\1:/g" #{dir}/**/*.rb" end

end

Pass extension to -i flag




def do_the_thing(dir) system "sed -E -i .tmp ’s/:([[:alnum:]]+)[[:space:]]=>/\\1:/g" #{dir}/**/*.rb" end

end

Spec Fails!





end

Passing an empty extension again





end

Spec Passes!





end

We know what this does!

Section V: Operation






end





private

def substitute_command 's/:([[:alnum:]]+)[[:space:]]=>/\\1:/g' end

end




def do_the_thing(dir) system "sed -E -i '' #{substitute_command} #{dir}/**/*.rb" end

private


end





private


end

Spec Passes!





private


end Really old regex syntax





private

def substitute_command ’s/:(\w+)\s=>/\\1:/g’ end

end New regex syntax





private

def substitute_command ’s/:(\w+)\s=>/\\1:/g’ end

end Spec Fails!





private


end Back to the old regex syntax





private


end

Spec Passes!

Section VI: Recovery


What do I need to do after refactoring?


Need to know two things decisively



Need to know two things decisively 1) Does the behavior still exist?



Need to know two things decisively 1) Does the behavior still exist?

2) Is it connected correctly?



What about QA?



Ideally, QA should find nothing

What if something goes wrong?




1) Take responsibility - what your code does in production is your responsibility




2) Evaluate risk of another change (a change to a change is still a change)




2) Evaluate risk of another change (a change to a change is still a change)

3) Fix the problem (if you can’t, find someone who can!)

Thank You!


Nell Shamrell-Harrington

Software Development Engineer at Chef

@nellshamrell

www.nellshamrell.com

[email protected]

http://www.nellshamrell.com

mailto:[email protected]

Thank You!


Nell Shamrell-Harrington

Software Development Engineer at Chef

@nellshamrell

www.nellshamrell.com

[email protected]

Thank you Jim and Jen!!!!

http://www.nellshamrell.com

mailto:[email protected]

first do no harm: surgical refactoring

Technology