淺談 groovy 與 aws 雲端應用開發整合

Groovy & AWS!雲端應⽤用開發整合

林彥宏!思創軟體 Senior Software Developer!

lyhcode.info / [email protected]

3

Polyglot Programming

for Java developers

Scala

Clojure

JRuby

Jython

Rhino

Groovy

http://VB.NET

4

A language that doesn't affect the way you think about programming, is not worth knowing.

— Alan Perlis

“

5

Hello, Groovy.

very good and enjoyable

fashionable, modern, and fun

6

2003

1.0

2007

designed by James Strachan

project manager Guillaume LaForge

glaforge.appspot.com

2.0

2012

https://github.com/groovy/groovy-coreApache License, Version 2.0

3.0

?

JSR-241Groovy – A New Standard Programming Language for the Java Platform

2004

2.3.5 2.4.0-beta

Groovy History

J2SE 5.0 Java SE 6 Java SE 7

1.820112006

JAX 2007

Innovation Awards

2008 2009

1995Spring, VMware

http://glaforge.appspot.com

7

JSR-223 Scripting for the Java Platform

JSR-241 The Groovy Programming Language

Groovy for Java Developers• Scripting language • Almost-zero learning

curve • More productive • Get things done faster,

better, and more easily

8

9本投影⽚片所包含的商標與⽂文字皆屬原著作者所有

10

Java

Groovy

Groovy Second language to learn after Java.

Gradle A better way to build your Java (EE) projects.

Geb The best way to test your web applications automatically.

Language

Development Tools

Framework

Grails Rapid web development framework based on Groovy and Java.

INSTALL GROOVY

11

GVM• Groovy enVironment Manager !

• Like RVM for Ruby or NVM for Node.js

12

Install GVM

13

# install gvm

curl -s get.gvmtool.net | bash

Platform supported:

• Linux

• Mac OSX

• Windows (with Cygwin)

• Solaris

• FreeBSD

Using GVM

14

gvm install groovy 2.3.4

gvm install grails 2.4.2

gvm install gradle 2.0

candidate version

15

gvm ls groovy

gvm use groovy 2.3.4

GETTING STARTED

16

17

groovy the interpreter

groovyc the compiler

groovysh Groovy Shell, REPL (Read-eval-print loop)

groovyConsole GUI editor for groovy

Compiling Groovy code to Java bytecode

18

Groovy Console

19

Groovy Web Consolegroovyconsole.appspot.com

20

Code Editor

v.s.

IDE

1. Groovy / Grails Tool Suite (Eclipse-based)

2. NetBeans 6.5+ 3. IntelliJ IDEA

1. Vim 2. Sublime Text 3. Gedit

Hello World in Java

21

1. public class HelloWorld {

2. public static void main(String[] args) {

3. System.out.println("Hello! World!");

4. }

5. }

6.

From Java To Groovy

22

1. public class HelloWorld {

2. public static void main(String[] args) {

3. System.out.println("Hello! World!");

4. }

5. }

6.

Hello World in Groovy

23

1. println "Hello! World!"

2.

Create Object

24

1. class Person {

2. def name

3. def age

4. }

5.

6. def john = new Person(name: 'John', age: 35)

7. println john.name

8.

dynamic typing

GString

25

1. println "Name: ${john.name}, Age: ${john.age}"

2.

3. println "Birth year: ${2014 - john.age}"

4.

Regex (regular expression)

26

1. pattern = /[0-9]{4}\-[0-9]{2}-[0-9]{2}/

2.

3. assert "2014-08-01" ==~ pattern

4.

5. // Assertion failed

6. assert "103-08-01" ==~ pattern

7.

Closure

27

1. def add = { x, y ->

2. x + y

3. }

4.

5. def z

6.

7. z = add 1, 2

8.

GDK (Groovy JDK)

28

1. new File('input.txt').text

2.

3. new URL('http://java.com/').text

4.

GDK (Groovy JDK)

29

1. new File('/tmp').eachFileMatch(~/.*\.sock/) {

2. file ->

3. println file.absolutePath

4. }

5.

Metaprogramming

30

1. String.metaClass.hello = {

2. "Hello ${delegate}"

3. }

4.

5. // Say Hello to John

6. "John".hello()

7.

Metaprogramming

31

1. String.metaClass.escapeHtml = {

2. delegate

3. .replaceAll('<', '<')

4. .replaceAll('>', '>')

5. .replaceAll(' ', ' ')

6. }

7.

8. "<p>line1<br />line2</p>".escapeHtml()

9.

List

32

1. def books = ["Java Cookbook", "Java in a Nutshell"]

2.

3. assert books instanceof ArrayList //;true

4.

5. println books.size()

6. println books[0]

7.

8. books << "Functional Programming in Java"

9.

10. books.each { println it }

11.

Map

33

1. def votes = [java: 9, groovy: 12, scala: 7]

2.

3. assert votes instanceof HashMap //;true

4.

5. votes.jruby = 5

6. votes << [clojure: 11]

7.

8. println votes.size()

9. println votes.java

10.

11. votes.each { println "${it.key} = ${it.value}" }

12.

34

1. ArrayList list = [1, 2, 3] + [4, 5, 6]

2.

3. list += [7, 8]

4. list << 9

5.

6.

7. HashMap map = [a: 1, b: 2, c: 3] + [d: 4, e: 5]

8.

9. map += [f: 6, g: 7]

10. map << [h: 8]

11.

JSON Support• Groovy 1.8 introduces a build-in JSON builder

and parser

!groovy.json.JsonSlurper

groovy.json.JsonBuilder

35

Using JsonSlurper

36

1. import groovy.json.*

2.

3. def result = new JsonSlurper().parseText('''

4. {

5. "first_name": "John",

6. "last_name": "Smith"

7. }

8. ''')

9.

10. result.first_name

11. result.last_name

12.

sample.json

37

1. {

2. "events": [

3. {

4. "name": "2014 Java Developer Day",

5. "date": "2014/08/01"

6. },

7. {

8. "name": "JCConf TW 2014",

9. "date": "2014/11/15"

10. }

11. ]

12. }

13.

Reading JSON

38

1. import groovy.json.*

2.

3. def payload = new File('sample.json')

4. // or new URL(‘http://.../sample.json’)

5.

6. def slurper = new JsonSlurper()

7. def result = slurper.parse(payload, 'UTF-8')

8.

Reading JSON

39

1. result.events.size()

2. //;;=> 2

3.

4. result.events.first().name

5. //;;=> 2014 Java Developer Day

6.

7. // List all events

8. result.events.each {

9. println it.name

10. }

11.

Building JSON

40

1. def builder = new groovy.json.JsonBuilder()

2. def root = builder.people {

3. person {

4. firstName 'John'

5. lastName 'Smith'

6. }

7. }

8.

9. builder.toPrettyString()

10.

{ "people": { "person": { "firstName": "John", "lastName": "Smith" } } } program output

41

Groovy and DSL (Domain-Specific Languages)

Groovy DSL• DSL: Domain-Specific Language • Could be implemented with Groovy Builders

42

sendmail {

from me

to someone

message “leave a message”

} (pseudo code)

AntBuilder

43

1. new AntBuilder().zip(

2. destFile: 'file1.zip',

3. basedir: 'dir'

4. )

5.

Using AntBuilder to compress a folder

44

1. new AntBuilder().mail(

2. mailhost: 'smtp.gmail.com',

3. mailport: 465, ssl: true,

4. user: 'username', password: 'password',

5. messagemimetype: 'text/html',

6. subject: 'Test Sending E-Mail with AntBuilder')

7. {

8. from name: 'AntBuilder', address: 'john@company'

9. to address: 'jack@company'

10. cc address: 'jackson@company'

11. message "<p><b>Test</b> only.</p>"

12. }

Using AntBuilder to send a email

GRADLEDevelopment Tools

45

Gradle• Build automation for Java/Java EE/Groovy

projects. • Based on Groovy.

46

47

Apache Ant

XML (build.xml)

Apache Maven

XML (pom.xml)

Gradle

Groovy DSL (build.gradle)

Define a Gradle Task

48

1. task hello {

2. doLast {

3. println "Hello"

4. }

5. }

6.

build.gradle

gradle hello

Default Java Project Layout

49

src/main/ java groovy resources webapp

src/test/ java groovy resources

build.gradle

Build a Java project

50

1. apply plugin: 'java'

2.

3. repositories {

4. mavenCentral()

5. }

6.

7. dependencies {

8. compile 'commons-codec:commons-codec:1.8'

9. }

10.

Use the Maven Central Repository

Dependencies Management

Execute a Gradle Task

51

gradle build

gradle test

gradle war

gradle jettyRun

gradle javadoc

task command

Gradle Plugins

52


2. apply plugin: 'groovy'

3. apply plugin: 'scala'

4. apply plugin: 'war'

5. apply plugin: 'maven'

6. apply plugin: 'wrapper'

7. apply plugin: 'application'

8. apply plugin: 'jetty'


10. apply plugin: 'eclipse'

11. apply plugin: 'idea'

12. apply plugin: 'codenarc'

13. apply plugin: 'checkstyle'

14. // and more

15.

Gradle Wrapper

53

./gradlew build

gradlew.bat build

For Linux, Mac OS X users…

For Windows users…

Gradle Test Report

54

GEBDevelopment Tools

55

Geb• A browser automation solution. • Based on Groovy and Selenium WebDriver.

56

Getting Started

57

1. @Grapes([

2. @Grab('org.gebish:geb-core:0.9.2'),

3. @Grab('org.seleniumhq.selenium:selenium-firefox-driver:2.42.0'),

4. @Grab('org.seleniumhq.selenium:selenium-support:2.42.0')

5. ])

6. import geb.Browser

!8. Browser.drive {

9. go "http://www.google.com.tw/"

10. }

11.

Geb Navigator API

58

1. println $('h3.heading').text()

2. $('input', name: 'fullName').value('user1')

3. $('button#btnSubmit').click()

4.

1. <h3 class=“heading”>…</h3>

2. <form action=“…”>

3. <input name=“fullName” value=“” />

4. <button id=“btnSubmit” class=“btn-primary”>Submit</button>

5. </form>HTML

Geb

Browser Automation

59

1. Browser.drive {

2. go 'https://www.google.com.tw/'

3.

4. $('input', name: 'q').value('CodeData')

5. $('input', name: 'btnI').click()

6.

7. waitFor { title.endsWith('CodeData') }

8.

9. println $('div.article h3 a').text()

10. }

11.

GROOVY SCRIPTING

60

61

Groovy Scripts

Java Library

Command-line Tools

Run with Shell or Cron jobs

Execute a Groovy Script

62

groovy myscript.groovy arg1

myscript arg1

Execute Groovy as Shell Script1. #!/usr/bin/env groovy

2.

3. args.each {

4. println "arg: ${it}"

5. }

6.

63

chmod a+x myscript

./myscript arg1

!

file name: myscript

DIVE INTO AWS

64

65

S3 (Simple Storage Service)

Elastic Beanstalk

EC2 (Elastic Compute Cloud)

RDS (Amazon Relational Database Service)

CloudFront (CDN)

Using AWS SDK with Groovy Script

Grapes and @Grab()

66

1. @Grab('com.amazonaws:aws-java-sdk:1.8.3')

2. import com.amazonaws.auth.*

3. import com.amazonaws.services.s3.*

4.

Grapes will do:

1. Resolving dependencies

2. Download packages (*.jar) from Maven repositories

group artifact

com.amazonaws:aws-java-sdk:1.8.3

version

67

aws-java-sdk-1.8.3.jar

commons-logging-1.1.1.jar httpclient-4.2.jar httpcore-4.2.jar commons-codec-1.3.jar jackson-core-2.1.1.jar jackson-databind-2.1.1.jar jackson-annotations-2.1.1.jar joda-time-2.3.jar

~/.groovy/grapes

depends on

All *.jar files will be stored in this folder

s3upload.groovy

68

1. def credentials = new BasicAWSCredentials(

2. accessKey,

3. secretKey

4. )

5.

6. // Get S3 client

7. def s3client = new AmazonS3Client(credentials)

8.

69

9. // Create a S3 Bucket

10. s3client.createBucket('s3.myhost')

11.

12. // Upload a file to S3

13. def localFile = new File('path-to-local-file')

!15. s3client.putObject(

16. 's3.myhost',

17. 'key-for-remote-file',

18. localFile

19. )

20.

WAR Deployment on AWS

70

Local WAR file

MyApp-1.0.war Amazon S3 Bucket

Amazon

Amazon Elastic BeansTalk !Application

upload Environmentdeploy

Create or update an environment

71

Accessing Elastic Beanstalk services with Groovy Script

+ Applications

eb_deploy.groovy

72

1. @Grab('com.amazonaws:aws-java-sdk:1.8.3')

2. import com.amazonaws.auth.*

3. import com.amazonaws.services.s3.*

4. import com.amazonaws.services.elasticbeanstalk.*

5. import com.amazonaws.services.elasticbeanstalk.model.*

6.

7. def credentials = new BasicAWSCredentials(accessKey, secretKey)

8.

9. def s3client = new AmazonS3Client(credentials)

10. def ebclient = new AWSElasticBeanstalkClient(credentials)

11.

73

12.

13. def bucket = ebclient.createStorageLocation().s3Bucket

14. s3client.putObject(bucket, objectKey, warFile)

15.

16. def request = new CreateApplicationVersionRequest()

17. request.applicationName = 'applicationName'

18. request.versionLabel = 'versionLabel'

19. request.description = 'description'

20. request.autoCreateApplication = true

21. request.sourceBundle = new S3Location(bucket, objectKey)

22. elasticBeanstalk.createApplicationVersion(request)

23.

24. request = new UpdateEnvironmentRequest()

25. request.environmentName = 'environmentName'

26. request.versionLabel = 'versionLabel'

27. elasticBeanstalk.updateEnvironment(request)

28.

① Upload .war to S3

② Create application version

③ Update environment

GRAILS FRAMEWORK

74

75

Grails Architecture

Grails

Groovy

Java Virtual Machine

Spring Hibernate SiteMesh Java EE

Create a Grails Project

76

grails create-app example

cd example

Run

77

grails run-app

grails -Dserver.port=8000 run-app

Specifies the HTTP port:

78

Browse http://localhost:8080/example

http://localhost:8080/example

79

BuildConfig

Config

BootStrap

DataSource

UrlMappings Tag LibraryFilter

Domain ClassView

ControllerService

Configuration MVC

Spock, JUnit

Testing

80

1. grails.project.dependency.resolution = { 2. repositories { 3. grailsPlugins() 4. grailsHome() 5. mavenLocal() 6. grailsCentral() 7. mavenCentral() 8. } 9. dependencies { 10. runtime 'mysql:mysql-connector-java:5.1.29' 11. } 12. plugins { 13. build ":tomcat:7.0.54" !15. compile ":scaffolding:2.1.2" 16. compile ':cache:1.1.7' 17. compile ":asset-pipeline:1.8.11" !19. runtime ":hibernate4:4.3.5.4" 20. runtime ":database-migration:1.4.0" 21. runtime ":jquery:1.11.1" 22. } 23. }

grails-app/conf/BuildConfig.groovy

81

GORM

Grails Object Relational Mapping

DataSource.groovy

RDBMS

Domain Class

82

grails create-domain-class User

1. class User {

2. String firstName

3. String lastName

4. Integer age

5.

6. static constraints = {

7. age range: 0..99

8. }

9. }

10.

83

1. @TestFor(User)

2. class UserSpec extends Specification {

3. void "The new user must have first and last names"() {

4. when:

5. def user = new User()

6. then:

7. !user.validate()

8. when:

9. user = new User(firstName: 'John', lastName: 'Smith')

10. then:

11. user.validate()

12. }

13. } grails test-app

Test Report

84

85

grails create-controller User

1. class UserController {

2. def index() {

3. redirect action: 'list'

4. }

5. def list() {

6. [

7. users: User.list()

8. ]

9. }

10. }

86

Views .gsp

+ =SiteMesh Layout .gsp

Main content area and additional head tags.

Theme, libraries and full page layout.

Final HTML output to browsers.

Decorated HTML

TagLib

Using custom tags in views.

87

1. <!DOCTYPE html>

2. <html>

3. <head>

4. <meta name="layout" content="bootstrap3"/>

5. <title>…</title>

6. </head>

7. <body>

8. <p>Totals: ${users.size()}</p>

9. </body>

10. </html>

grails-app/views/user/list.gsp

SiteMesh

88

1. <!DOCTYPE html>

2. <html>

3. <head>

4. <title>…</title>

5. <link rel="stylesheet" href="css/bootstrap.min.css">

6. <link rel="stylesheet" href="css/bootstrap-theme.min.css">

7. <script src="js/bootstrap.min.js"></script>

8. <g:layoutHead/>

9. </head>

10. <body>

11. <g:layoutBody/>

12. </body>

13. </html>

grails-app/views/layout/bootstrap3.gsp

89

1. class BootstrapTagLib {

2. static namespace = "bs3"

3. def alert = { attrs, body ->

4. out << """<div class="alert alert-${attrs.type?:'success'}">"""

5. out << body()

6. out << '</div>'

7. }

8. }

9.

1. <bs3:alert type="warning">Warning messages.</bs3:alert>

2.

3.

90

http://localhost:8080/project/controller/action/ID

Default URL Mapping


2. def show(long id) {

3. [user: User.get(id)]

4. }

5. }

…/user/show/1

91

localhost:8080/user/list.html


2. …

3. def list() {

4. def users = User.list()

5. withFormat {

6. html users: users

7. xml { render users as XML }

8. json { render users as JSON }

9. }

10. }

11. }

localhost:8080/user/list.xml

localhost:8080/user/list.json

Multiple Response Content-Type

Grails Application Packaging

92

grails war

93

Grails Web Application

Elastic Beanstalk

RDS

EC2

project-1.0.war

Database

1. MySQL

2. PostgreSQL

3. Oracle

4. SQL Server

Pre-configured VMs

1. Linux

2. Java

3. TomcatVirtual Machine

Install Elastic Beanstalk Plugin

94

1. grails.project.dependency.resolution = { 2. … 3. plugins { 4. … 5. compile ':aws-elastic-beanstalk:0.3' 6. } 7. } 8.


Setup Elastic Beanstalk Plugin

95

grails-app/conf/Config.groovy

1. grails { 2. plugin { 3. awsElasticBeanstalk { 4. applicationName = 'myApplication' 5. environmentName = 'myEnvironment' 6. savedConfigurationName = 'default' 7. systemProperties = [ 8. 'property.name.1': 'property-value-1', 9. 'property.name.2': 'property-value-2'] 10. } 11. } 12. } 13.

96


1. environments { 2. production { 3. grails.logging.jul.usebridge = false 4. grails.serverURL = "http://{environment_ name}.elasticbeanstalk.com" 5. } 6. } 7.

97

grails-app/conf/DataSource.groovy

1. production { 2. dataSource { 3. username = "{user}" 4. password = "{password}" 5. pooled = true 6. dbCreate = "update" 7. driverClassName = "com.mysql.jdbc.Driver" 8. url = "jdbc:mysql://{endpoint}:{port_number}/ebdb" 9. dialect = org.hibernate.dialect.MySQL5InnoDBDialect 10. properties { 11. validationQuery = "SELECT 1" 12. testOnBorrow = true 13. testOnReturn = true 14. testWhileIdle = true 15. timeBetweenEvictionRunsMillis = 1800000 16. numTestsPerEvictionRun = 3 17. minEvictableIdleTimeMillis = 1800000 18. } 19. } 20. } 21.

xxxxxxxxxxxxxxx

!2. dataSource {

!4. def prop = System.properties

!6. username = "${prop.RDS_USERNAME}" 7. password = "${prop.RDS_PASSWORD}"

!9. url = "jdbc:mysql://${prop.RDS_HOSTNAME}:${prop.RDS_PORT}/ebdb" 10. … 11. } 12.

98

1. RDS_USERNAME 2. RDS_PASSWORD 3. RDS_HOSTNAME 4. RDS_PORT 5. RDS_DB_NAME

RDS System Properties

Deploy to Elastic Beanstalk

99

grails aws-eb-deploy

100

EC2 InstanceLoad Balancer

Amazon CloudFront

Amazon S3 Bucket

Web Application Server

Assets (CSS, JavaScript, Images)CDN (Content delivery network)

Deploy Grails web application assets to S3 and CloudFront.

CDN Asset Pipeline Grails Plugin

101

1. grails.project.dependency.resolution = { 2. … 3. plugins { 4. … 5. compile ':cdn-asset-pipeline:0.3.5' 6. } 7. } 8.


CDN Asset Pipeline Grails Plugin

102

1. grails { 2. assets { 3. cdn { 4. provider = 's3' 5. directory = 'my-bucket' 6. accessKey = '{MY_S3_ACCESS_KEY}' 7. secretKey = '{MY_S3_SECRET_KEY}' 8. storagePath = "assets/${appName}-${appVersion}/" 9. expires = 365 10. gzip = true 11. } 12. } 13. } 14.


Use the {version} value to prevent outdated CDN cache.

Use CDN-based Assets

103

1. // S3 Only 2. grails.assets.url = "https://{bucketName}.s3.amazonaws.com/

my-bucket/assets/${appName}-${appVersion}” 3. 4. // S3 + CloudFront 5. grails.assets.url = "https://{identity}.cloudfront.net/

my-bucket/assets/${appName}-${appVersion}" 6.


104

Use Groovy, Gradle or Grails for your next project.

Learn More

105

Search CodeData+Groovy

106

THANK YOU

[email protected]&A

淺談 groovy 與 aws 雲端應用開發整合

Presentations & Public Speaking