6/8/2010
Instructor: Jeff Payne
Why Software Fails
Connectivity/Extensibility/Complexity
Software is different from everything we create in the physical world
Software quality is poorly measured and managed
What is real QA?
Some traditional methods dont mention quality, software, value to the customer
Verification - Old school QA - have we built the product right?
Conforming with standards, following the process, tracing back artifacts to requirements
Process oriented/process centric
Only do the parts that you think have value
dont require much software knowledge
Validation - have we built the right product?
validate against requirements document
require much more software expertise than verification (you have to be technical)
Risk Management & QA
Focus on critical components
Risk Matrix: consequence vs. likelihood
Metrics
What is the purpose of collecting metrics?
Goal Question Metric (GQM) (google it)
Defect Metrics: defects produced, corrected, rates
good because they get to the heart of the matter: does the software work
how close are we to being done
Risk Metrics: how many risks have been identified/mitigated etc, code coverage
Productivity/rework metrics
velocity: how much did we get done in an iteration (measure per team not person)
rework: time & effort spent fixing defects
time spent building new things vs fixing old things
Relative metrics and trends better than absolute and individual numbers
Tuesday, June 8, 2010
Mastering Design Patterns
6/8/2010
Instructor: Robert Martin
...A discussion on Hubble and the expansion and age of the universe
Design Patterns book 1995 "Gang of Four" GoF95
Design patterns are encoded knowledge given a name and a canonical form: once you know/recognize it you can stop thinking about it
A design pattern is a solution to a recurring problem
Abstract Server pattern: so simple it's not in the books
or polymorphism pattern
Clients and the interfaces they used get grouped together, NOT grouped by hierarchies
Adapted Server pattern
Adapter allows you to "adapt" a class you dont own the source code of
Can combine incompatible interfaces/convert one interface to another
A delegation relationship is when the Adapter simply passes on the call to the connected interface
Class form of the adapter pattern: the adapter inherits from both the interface and the server
Template Method Pattern (or write a loop once)
Strategy Pattern
It is polite to change the name of the class to include the design pattern name to aid recognition for the reader
Data structures have well known data and very little behavior (methods)
Objects have well known behaviors and very little known data
...a discussion on the early expansion of the universe shortly after the big bang
State Pattern
How do we implement finite state machines?
Implementing in a huge nested switch statement is very hard to maintain
SMC (google it): implementation of state machine classes is generateble if you know the state event/action table
Parsers, network protocols, GUIs, are FSMs
Command Pattern
Remarkably simple but the repercussions are enormous
A way to decouple what gets done from who does it
A sensor can be decoupled from the command it calls
Can also be used to store a list of commands and "undo" them as necessary
Or transaction commands in a DB which can be executed later during downtime
Decoupling what gets done from WHEN it gets done
A list of commands (some of which add new commands to the list) can be used for multitasking/task switching on a single stack (called run to completion) There's no blocking: if a command needs to wait for an event, it must clone itself and put it on the list
...a discussion on stars and supernovas/hypernovas
Composite Pattern
Similar to the command pattern & very simple
A way to take many objects and treat them as one
The composite exists at a lower level so the higher level model is not affected
"take all details and shove them down as far as you can"
dont expose details at the highest levels
If you're doing to treat a composite of objects identically, then a composite is useful. If you're going to scan the objects and treat them different, the composite pattern cannot be used
Proxy Pattern
Can be used to create a "proxy" class that hides dependencies and isolates an application from (for instance) a DB. The proxy database interface layer depends on the app and the DB: the app does not depend on the DB because the dependencies have been inverted
Can also be used for cross-process communication
However: the proxy objects are complicated & difficult to maintain (but at least all your complications are isolated in the proxy)
Observer Pattern
(...a discussion on leap years and leap seconds and the changing rotation of the earth)
Allows an object to notify multiple dependent objects every time it changes state
small changes can be pushed thru the notify() function
larger changes need to be pulled from the dependent object when they are needed
sometimes hints can be passed to narrow down where the change occurred
Can build systems that watch & react to other objects as they change: a cool form of indirection until you have to debug it.
Comes up a lot in GUI's, callbacks, listeners, MVC
MVC was probably the first design pattern to get a name (70s)
The input/process/output split is the same, but the object/observer relation is not the same as the original smalltalk version of MVC
What does MVC mean anymore? not much
Document View Pattern is similar to MVC
merges the view/controller
Model View Presenter
How do you write tests against a GUI?
Move complexity out of the view into a presenter
Presenter observes the model
View queries the presenter
Presenter restructures the data from the model so it is presentable
View is stupid
Presenter knows nothing about the GUI: it is creating data structures
Now you can test the presenter! And the view is so stupid you don't need to test it, or the test is trivial
Instructor: Robert Martin
...A discussion on Hubble and the expansion and age of the universe
Design Patterns book 1995 "Gang of Four" GoF95
Design patterns are encoded knowledge given a name and a canonical form: once you know/recognize it you can stop thinking about it
A design pattern is a solution to a recurring problem
Abstract Server pattern: so simple it's not in the books
or polymorphism pattern
Clients and the interfaces they used get grouped together, NOT grouped by hierarchies
Adapted Server pattern
Adapter allows you to "adapt" a class you dont own the source code of
Can combine incompatible interfaces/convert one interface to another
A delegation relationship is when the Adapter simply passes on the call to the connected interface
Class form of the adapter pattern: the adapter inherits from both the interface and the server
Template Method Pattern (or write a loop once)
Strategy Pattern
It is polite to change the name of the class to include the design pattern name to aid recognition for the reader
Data structures have well known data and very little behavior (methods)
Objects have well known behaviors and very little known data
...a discussion on the early expansion of the universe shortly after the big bang
State Pattern
How do we implement finite state machines?
Implementing in a huge nested switch statement is very hard to maintain
SMC (google it): implementation of state machine classes is generateble if you know the state event/action table
Parsers, network protocols, GUIs, are FSMs
Command Pattern
Remarkably simple but the repercussions are enormous
A way to decouple what gets done from who does it
A sensor can be decoupled from the command it calls
Can also be used to store a list of commands and "undo" them as necessary
Or transaction commands in a DB which can be executed later during downtime
Decoupling what gets done from WHEN it gets done
A list of commands (some of which add new commands to the list) can be used for multitasking/task switching on a single stack (called run to completion) There's no blocking: if a command needs to wait for an event, it must clone itself and put it on the list
...a discussion on stars and supernovas/hypernovas
Composite Pattern
Similar to the command pattern & very simple
A way to take many objects and treat them as one
The composite exists at a lower level so the higher level model is not affected
"take all details and shove them down as far as you can"
dont expose details at the highest levels
If you're doing to treat a composite of objects identically, then a composite is useful. If you're going to scan the objects and treat them different, the composite pattern cannot be used
Proxy Pattern
Can be used to create a "proxy" class that hides dependencies and isolates an application from (for instance) a DB. The proxy database interface layer depends on the app and the DB: the app does not depend on the DB because the dependencies have been inverted
Can also be used for cross-process communication
However: the proxy objects are complicated & difficult to maintain (but at least all your complications are isolated in the proxy)
Observer Pattern
(...a discussion on leap years and leap seconds and the changing rotation of the earth)
Allows an object to notify multiple dependent objects every time it changes state
small changes can be pushed thru the notify() function
larger changes need to be pulled from the dependent object when they are needed
sometimes hints can be passed to narrow down where the change occurred
Can build systems that watch & react to other objects as they change: a cool form of indirection until you have to debug it.
Comes up a lot in GUI's, callbacks, listeners, MVC
MVC was probably the first design pattern to get a name (70s)
The input/process/output split is the same, but the object/observer relation is not the same as the original smalltalk version of MVC
What does MVC mean anymore? not much
Document View Pattern is similar to MVC
merges the view/controller
Model View Presenter
How do you write tests against a GUI?
Move complexity out of the view into a presenter
Presenter observes the model
View queries the presenter
Presenter restructures the data from the model so it is presentable
View is stupid
Presenter knows nothing about the GUI: it is creating data structures
Now you can test the presenter! And the view is so stupid you don't need to test it, or the test is trivial
SOLID Principles of Object-Oriented System Design
6/7/2010
Instructor: Robert Martin
a discussion on determining the size of the galaxy using globular clusters and variable stars
a discussion on programming languages
C > Java > ruby > functional languages (f#, scala, closure)
functional languages: scalable/multiple threads: moore's law has died
no locking problems
why are all our languages OO?
What can OO do that C cant?
OO is a way to manage dependencies between pieces of source code
polymorphism allows us to invert the dependency of modules
you can do this in C with function pointers, but it's dangerous
Structured Programming - Djikstra "goto may be considered harmful"
discipline applied to direct forms of control - structured
discipline applied to indirect forms of control - OO
What causes rotten software? Complexity
Fragility: breaks easily, can't anticipate consequences of a change
Rigidity: Every change forces a host of other changes
Reusability: desirable things that a module does cant be untangled from the undesirable things. It's too easy to make dependencies on things we don't need
All these things are brought about by bad dependencies
Case study: copy routine
copy characters from the keyboard to the printer
copy depends on keyboard/printer
flow of control flows with dependencies:
void copy() {
int c;
while((c=kbd()!=EOF) print(c);
}
later... add support for other input/output devices
flow of control goes opposite of dependencies:
void copy() {
int c;
while(c=getchar()!=eof) putchar(c);
}
getchar/putchar uses stdin/stdout by default
it can be maintained because "the code can already do that"
Single Responsibility Principle
A class should only have 1 reason to change
Group together things that change for the same reasons
Separate things that change for different reasons
Open/Closed Principle
Add new functionality by adding new code, not editing old code
A module should be open for extension but closed for modification
A discussion on measuring the distance to the sun
What do you do when your abstractions get in the way?
the wrong abstractions cannot protect you from unanticipated changes
Agility can be a strategy to protect you: put in almost no abstractions at first
Show code to customer as early as possible and reactively add abstractions
This is a strategy but it is not perfect
One of the keys to making OO software work is TDD
Nothing makes a system more flexible than a suite of tests (by an order of magnitude)
If you have a bad design, and a suite of tests, you are not afraid to improve the design
Therefore tests are more important than a good design
QA should be writing acceptance tests at a high level (at the frontend of the dev process)
Developers should be writing unit tests at a lower level as development progresses
Liskov Substitution Principle
All derived classes must be substitutable for their base classes
Every substitution violation eventually leads to an open/closed violation
Inheritance does not mean "is a"
Inheritance means the variables/functions of the base class are redefined in the inherited class
The fact that a square is a rectangle does not mean that the square inherits from a rectangle
Taking behavior away from the base class is the core problem. The user has a right to expect that behavior.
Dependency Inversion Principle
OO programs are programs where the flow of control is opposed by the source code dependencies
OO programs protect you from the creation of new derivative types
When you add new functions to the hierarchy, the user is not protected
When you add new data structures/types to the hierarchy, the user is protected
Procedural programs protect you from the creation of new functions
When you add new functions to the hierarchy, the user is not protected
When you add new data structures/types to the hierarchy, the user is protected
Dependency Inversion principle:
If A depends on B: B should be abstract
Java: interfaces
C++: pure virtual functions
Can you really do that all the time? no. (for example can't "new" an abstract class.)
Discussion of smalltalk vs C++ re:type-safety
TDD can replace type-safety (dont need the compiler to check, the tests can check)
This led to Ruby/Python
Whats bad about overriding a function that has an implementation?
Should be adding functionality, not taking away
If you depend on a derived class, you depend on everything the base class depends on
Inheritance is a very strong relationship: use with care
But inheritance is what gives us polymorphism, which allows this dependency inversion
This is why you should inherit from abstract classes to reduce the dependencies
Other languages (python/ruby) can do polymorphism without inheritance. This makes python/ruby code much easier to write.
A definition of a programmer is someone who is constantly learning. Every programmer should learn a new language every year just on principle.
This class is actually principles for statically typed OO languages (java/C++). The same principles apply in other languages, but in different ways.
"All solutions are short term"
Thinks Ruby is the next language to crest, then closure
Discussion on where gold comes from? Supernovas
Interface Segregation Principle
A fat class is a class with many methods and many users. A new user comes along and wants to add a new function to the fat class. All the other users get the new function too, so you have a massive recompilation and deployment.
Find groupings of methods that are only used by certain users. Create abstract interface clsses for each grouping. Have the fat class multiply inherit from the abstract interface classes.
Design is about manipulating code: if you can't envision the code you are not doing design
Comments are lies: they are far away from the things that they describe
Comments: only write them if you have to. Comments are a failure to make yourself clear in the code.
The worst kind of comment is commented-out code. The old code is in the VCS, if you need it you can go get it.
Functions should be max 4-5 lines long
Journalists work with the rule that the most important stuff should be at the top. Each line is more and more detailed. Code should be organized the same way so that it is easy to read/understand.
How many arguments should be passed into a function? Zero is best. Three is getting tough. If you are passing 5 things, you should probably be passing an object.
The worst thing you can pass into a function is a bool: you are announcing that the function does two things. Split it into two functions
Catching output in arguments is also bad, forces reader to do a doubletake
Instructor: Robert Martin
a discussion on determining the size of the galaxy using globular clusters and variable stars
a discussion on programming languages
C > Java > ruby > functional languages (f#, scala, closure)
functional languages: scalable/multiple threads: moore's law has died
no locking problems
why are all our languages OO?
What can OO do that C cant?
OO is a way to manage dependencies between pieces of source code
polymorphism allows us to invert the dependency of modules
you can do this in C with function pointers, but it's dangerous
Structured Programming - Djikstra "goto may be considered harmful"
discipline applied to direct forms of control - structured
discipline applied to indirect forms of control - OO
What causes rotten software? Complexity
Fragility: breaks easily, can't anticipate consequences of a change
Rigidity: Every change forces a host of other changes
Reusability: desirable things that a module does cant be untangled from the undesirable things. It's too easy to make dependencies on things we don't need
All these things are brought about by bad dependencies
Case study: copy routine
copy characters from the keyboard to the printer
copy depends on keyboard/printer
flow of control flows with dependencies:
void copy() {
int c;
while((c=kbd()!=EOF) print(c);
}
later... add support for other input/output devices
flow of control goes opposite of dependencies:
void copy() {
int c;
while(c=getchar()!=eof) putchar(c);
}
getchar/putchar uses stdin/stdout by default
it can be maintained because "the code can already do that"
Single Responsibility Principle
A class should only have 1 reason to change
Group together things that change for the same reasons
Separate things that change for different reasons
Open/Closed Principle
Add new functionality by adding new code, not editing old code
A module should be open for extension but closed for modification
A discussion on measuring the distance to the sun
What do you do when your abstractions get in the way?
the wrong abstractions cannot protect you from unanticipated changes
Agility can be a strategy to protect you: put in almost no abstractions at first
Show code to customer as early as possible and reactively add abstractions
This is a strategy but it is not perfect
One of the keys to making OO software work is TDD
Nothing makes a system more flexible than a suite of tests (by an order of magnitude)
If you have a bad design, and a suite of tests, you are not afraid to improve the design
Therefore tests are more important than a good design
QA should be writing acceptance tests at a high level (at the frontend of the dev process)
Developers should be writing unit tests at a lower level as development progresses
Liskov Substitution Principle
All derived classes must be substitutable for their base classes
Every substitution violation eventually leads to an open/closed violation
Inheritance does not mean "is a"
Inheritance means the variables/functions of the base class are redefined in the inherited class
The fact that a square is a rectangle does not mean that the square inherits from a rectangle
Taking behavior away from the base class is the core problem. The user has a right to expect that behavior.
Dependency Inversion Principle
OO programs are programs where the flow of control is opposed by the source code dependencies
OO programs protect you from the creation of new derivative types
When you add new functions to the hierarchy, the user is not protected
When you add new data structures/types to the hierarchy, the user is protected
Procedural programs protect you from the creation of new functions
When you add new functions to the hierarchy, the user is not protected
When you add new data structures/types to the hierarchy, the user is protected
Dependency Inversion principle:
If A depends on B: B should be abstract
Java: interfaces
C++: pure virtual functions
Can you really do that all the time? no. (for example can't "new" an abstract class.)
Discussion of smalltalk vs C++ re:type-safety
TDD can replace type-safety (dont need the compiler to check, the tests can check)
This led to Ruby/Python
Whats bad about overriding a function that has an implementation?
Should be adding functionality, not taking away
If you depend on a derived class, you depend on everything the base class depends on
Inheritance is a very strong relationship: use with care
But inheritance is what gives us polymorphism, which allows this dependency inversion
This is why you should inherit from abstract classes to reduce the dependencies
Other languages (python/ruby) can do polymorphism without inheritance. This makes python/ruby code much easier to write.
A definition of a programmer is someone who is constantly learning. Every programmer should learn a new language every year just on principle.
This class is actually principles for statically typed OO languages (java/C++). The same principles apply in other languages, but in different ways.
"All solutions are short term"
Thinks Ruby is the next language to crest, then closure
Discussion on where gold comes from? Supernovas
Interface Segregation Principle
A fat class is a class with many methods and many users. A new user comes along and wants to add a new function to the fat class. All the other users get the new function too, so you have a massive recompilation and deployment.
Find groupings of methods that are only used by certain users. Create abstract interface clsses for each grouping. Have the fat class multiply inherit from the abstract interface classes.
Design is about manipulating code: if you can't envision the code you are not doing design
Comments are lies: they are far away from the things that they describe
Comments: only write them if you have to. Comments are a failure to make yourself clear in the code.
The worst kind of comment is commented-out code. The old code is in the VCS, if you need it you can go get it.
Functions should be max 4-5 lines long
Journalists work with the rule that the most important stuff should be at the top. Each line is more and more detailed. Code should be organized the same way so that it is easy to read/understand.
How many arguments should be passed into a function? Zero is best. Three is getting tough. If you are passing 5 things, you should probably be passing an object.
The worst thing you can pass into a function is a bool: you are announcing that the function does two things. Split it into two functions
Catching output in arguments is also bad, forces reader to do a doubletake
Tutorial: Test-driven Development
Instructor: Rob Myers
In the traditional software development cycle testing is done at the end of an implementation cycle. That is, one analyzes the problem, designs a solution, and then implements the results of analysis and design. After all is said and done the implementation is tested. Tests include unit, integration, system and acceptance.
Unit testing is the most basic and includes testing the fundamental units of functionality in a module (i.e. subroutines or classes).
Integration testing is one level up and includes testing the coupling of modules. System testing tests all the modules fitly framed together into a running program. Acceptance testing is further verification and validation testing.
Re-running all of these types of tests is called regression testing. Ideally regression testing is completely automated. Even more ideal is an automated build process coupled with automated regression testing. The best of all is a process attached to version control which checks out the code, runs the build, and then run the regression suite – on multiple platforms. This is called continuous integration.
Enter test-driven development…
TDD is an Agile practice for implementation and unit testing. It is a method of code development that includes writing a test first, then coding until that tests passes.
The recipe is:
1. Write one unit test.
2. Build or add to the fundamental unit under test until everything compiles.
3. Watch the test fail!
4. Make the unit test (and all previous unit tests) pass by changing the fundamental unit under test.
5. Refactor, refactor, refactor.
6. Repeat.
During the last part of the tutorial we paired up and practiced TDD. Rob had us implement a Set class. He outlined several requirements of the Set class then had teams follow the TDD recipe until all requirements were completed. My team was the first to finish :-). The guy I paired up with is used Visual Studio 2010 with C#. The .NET framework has built-in unit testing capabilities. (I have been using Google Test for my own Linux based C++ unit testing)
Rob suggests three books:
* Refactoring (Martin Fowler)
* Test-driven Development (Kent Beck)
* Working Effectively with Legacy Code (Michael Feathers)
In the traditional software development cycle testing is done at the end of an implementation cycle. That is, one analyzes the problem, designs a solution, and then implements the results of analysis and design. After all is said and done the implementation is tested. Tests include unit, integration, system and acceptance.
Unit testing is the most basic and includes testing the fundamental units of functionality in a module (i.e. subroutines or classes).
Integration testing is one level up and includes testing the coupling of modules. System testing tests all the modules fitly framed together into a running program. Acceptance testing is further verification and validation testing.
Re-running all of these types of tests is called regression testing. Ideally regression testing is completely automated. Even more ideal is an automated build process coupled with automated regression testing. The best of all is a process attached to version control which checks out the code, runs the build, and then run the regression suite – on multiple platforms. This is called continuous integration.
Enter test-driven development…
TDD is an Agile practice for implementation and unit testing. It is a method of code development that includes writing a test first, then coding until that tests passes.
The recipe is:
1. Write one unit test.
2. Build or add to the fundamental unit under test until everything compiles.
3. Watch the test fail!
4. Make the unit test (and all previous unit tests) pass by changing the fundamental unit under test.
5. Refactor, refactor, refactor.
6. Repeat.
During the last part of the tutorial we paired up and practiced TDD. Rob had us implement a Set class. He outlined several requirements of the Set class then had teams follow the TDD recipe until all requirements were completed. My team was the first to finish :-). The guy I paired up with is used Visual Studio 2010 with C#. The .NET framework has built-in unit testing capabilities. (I have been using Google Test for my own Linux based C++ unit testing)
Rob suggests three books:
* Refactoring (Martin Fowler)
* Test-driven Development (Kent Beck)
* Working Effectively with Legacy Code (Michael Feathers)
Tutorial: A guide to transitioning to Agile methods of software development.
Instructor was Mike Cohn.
Why is transitioning to Agile methods so hard?
Introducing agile is hard b/c it’s both top-down and bottom-up. I.e Vision from the top. Practiced by the ground up by people who see the benefits.
Best practices -> good practices in most contexts.
Organizations are complex adaptive systems … not a system of well defined behavior where “closing the gaps” happen in a known way.
Aside: SCRUM is the most popular agile process these days. However, Mike is a fan of unbranded agile methods.
So …. the agile process should be a good fit with an organizations environment. Measure good fit if we are producing higher quality software at a faster pace.
ADAPTING to Agile
A = awareness (that there is room for improvement)
D = desire (to change)
A = ability (to work in an agile manner)
P = promote (early success to build momentum)
T = transfer (the impacts of agile throughout the origination so it sticks)
ScrumMaster…lol
Individual and group change:
Individuals will move through the Awareness, Desire, and Ability stage at their own rate
Early adapter and leaders can Promote to build Awareness and Desire and will need to Transfer impacts of agile to other groups
Comparative agility web-site to check where your agility practices are against the industry.
Iterating toward agility
It’s not about getting to Agile; it’s about getting more Agile – about getting better.
Improvement communities (a.k.a. grouplets) (a.k.a. communities of practice) – a small group of people passionate about a given topic (i.e. testing, documentation, or something else). Does the work of how organization implement changes. Also focuses on goals of practical relevance.
Leadership
Not just managers, but opinion leaders, et al
Lead through example, questions, and focus: nudge, poke and prod
Overcoming Resistance
How they resist vs. Why they resist
How they resist: passive, active
Why they resist: skeptic, saboter
Why is transitioning to Agile methods so hard?
Introducing agile is hard b/c it’s both top-down and bottom-up. I.e Vision from the top. Practiced by the ground up by people who see the benefits.
Best practices -> good practices in most contexts.
Organizations are complex adaptive systems … not a system of well defined behavior where “closing the gaps” happen in a known way.
Aside: SCRUM is the most popular agile process these days. However, Mike is a fan of unbranded agile methods.
So …. the agile process should be a good fit with an organizations environment. Measure good fit if we are producing higher quality software at a faster pace.
ADAPTING to Agile
A = awareness (that there is room for improvement)
D = desire (to change)
A = ability (to work in an agile manner)
P = promote (early success to build momentum)
T = transfer (the impacts of agile throughout the origination so it sticks)
ScrumMaster…lol
Individual and group change:
Individuals will move through the Awareness, Desire, and Ability stage at their own rate
Early adapter and leaders can Promote to build Awareness and Desire and will need to Transfer impacts of agile to other groups
Comparative agility web-site to check where your agility practices are against the industry.
Iterating toward agility
It’s not about getting to Agile; it’s about getting more Agile – about getting better.
Improvement communities (a.k.a. grouplets) (a.k.a. communities of practice) – a small group of people passionate about a given topic (i.e. testing, documentation, or something else). Does the work of how organization implement changes. Also focuses on goals of practical relevance.
Leadership
Not just managers, but opinion leaders, et al
Lead through example, questions, and focus: nudge, poke and prod
Overcoming Resistance
How they resist vs. Why they resist
How they resist: passive, active
Why they resist: skeptic, saboter
Subscribe to:
Posts (Atom)