kafkawize documentation€¦ · apache kafka is widely being adapted in organizations irrespective...

45
Kafkawize Documentation Release 4.1 Read the Docs Apr 26, 2020

Upload: others

Post on 26-Apr-2020

7 views

Category:

Documents


0 download

TRANSCRIPT

Kafkawize DocumentationRelease 41

Read the Docs

Apr 26 2020

Contents

1 Introduction 311 What is Kafkawize 312 Background 313 Key features 414 41 glimpse 415 How is it developed 416 Architecture 417 Git Repositories 518 Developer 5

2 Getting Started 721 Demo 722 Prerequisites 723 1 Download Kafkawize 724 2 Download Metastore 725 3 License Key 826 4 Build KW Cluster Api Application 827 5 Configure Kafkawize UI Api Application 828 6 Metastore setup 929 7 Run KW and KWClusterApi 10210 8 Kafka Connectivity 11211 9 Final Check 11

3 Migration Kafkawize from 35 to 40 13

4 Features 1541 Security 1542 Teams and Role Management 1543 Login 1644 Dashboard 1645 Browse 1746 Requests 1847 Clusters 1948 Audit 2049 Approve Or Decline 22410 Users 23411 Teams 24

i

412 Synchronize Metadata 25413 Admin 26414 Profile 26415 Logout 26

5 Functionalities 27

6 Customize 29

7 Trouble shooting 31

8 Release Notes 33

9 Roadmap 35

10 Contact Me 37

11 Submit a Review 39

12 Indices and tables 41

ii

Kafkawize Documentation Release 41

Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization Read more

Contents

Contents 1

Kafkawize Documentation Release 41

2 Contents

CHAPTER 1

Introduction

11 What is Kafkawize

Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization

12 Background

Let me give some background Apache Kafka is widely being adapted in organizations irrespective of the scale Itrsquosunique features like scalability retention and reliability unlike the traditional messaging platforms makes it stand out

While adapting Kafka you would notice there are few manual activities like creating topics acls updating configu-ration etc There have been few topic management tools in the market which provide automation to certain extenthowever its not complete Assume there is a team requesting for a kafka topic with in an organization They wouldeither request for it through email with a template or any user interface The actual kafka team in the organization ex-ecutes the commands to create topics With Development Test Acceptance and Production environments managingthese configurations by the Kafka team and handling all the requests from various teams would become a hassle asyou see the growth of teams using Kafka It is just not manual activity but Audit time and effort the team has to putin

Apart from that this meta data (Kafka topics acls) which is stored in ZK could be easily stored in a good readableformat and can be used as source of truth

Hence I saw the need of Kafkawize ndash a self service topic management tool to keep ZERO interaction by other Teamswith Kafka team for topics acls or schemas creation by introducing roles and authorizations to users of teams It isdeveloped with a simple front end and backend with Spring boot It saves time and effort governs information avoidsrisk in kafka config loss avoids manual activities and mistakes maintains up to date SOT (source of truth)

3

Kafkawize Documentation Release 41

13 Key features

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

14 41 glimpse

15 How is it developed

Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it

16 Architecture

Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end

User Interface Api directly communicates between Frontend UI and Cluster API

Front end is built with AngularJs HTML and Java script

Cluster API acts as middle layer between Kafka brokers and UserInterface API

4 Chapter 1 Introduction

Kafkawize Documentation Release 41

Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client

Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth

Requests from users are directed to cluster api and also data is stored metastore

On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application

17 Git Repositories

UI Api httpsgithubcommuralibasanikafkawize

Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi

18 Developer

Muralidhar Basani

LinkedIn httpswwwlinkedincominmuralibasani

Web httpskafkawizecom

17 Git Repositories 5

Kafkawize Documentation Release 41

6 Chapter 1 Introduction

CHAPTER 2

Getting Started

21 Demo

Lets start with setting up Kafkawize on your WindowsLinuxUnix systems

22 Prerequisites

bull Any IDE like IntelliJNetbeans or Eclipse to build the applications

bull Java installed machines to run the applications

bull Rdbms (like OracleMySQL) or Apache Cassandra

23 1 Download Kafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi

OR

Download Kafkawize 41 bundle here Kafkawize_41

24 2 Download Metastore

If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload

If Rdbms is used as metastore download

Mysql httpsdevmysqlcomdownloads

7

Kafkawize Documentation Release 41

Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml

Or any other Rdbms is ok

25 3 License Key

License key is not required from 41 version

26 4 Build KW Cluster Api Application

This Api does the below

bull Receives and responds to calls from UI Api

bull Connects to Kafka Brokers with Kafka AdminClient Api

bull There is no connection to any metastore

bull Swagger enabled

Unzip kafkawizeclusterapizip and open in any IDE

Configure the application properties (srcmainresources) if port has to be changed else default port is 9343

and run maven command to create a runnable jar

mvn clean package

This should create a jar (kafkawizeclusterapi-41jar) in target dir

Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even

27 5 Configure Kafkawize UI Api Application

This Api does the below

bull Users interact with interface with this Api

bull All the end points in this application either connect to Metastore or Cluster Api or both

Unzip kafkawize-(version)zip and open in any IDE

Configure application properties

In kafkawizesrcmainresourcesapplicationproperties

default port is 9097 if port has to be changed else

serverport9097

If metastore is cassandra (from step 4)

setstore type as cassandra

customdbstoretype=cassandra

8 Chapter 2 Getting Started

Kafkawize Documentation Release 41

configure cassandra host port and keyspace

customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042

If metastore is rdbms (from step 4)

setstore type as rdbms

customdbstoretype=rdbms

bull Install and run Rdbms (like MysqlOracle) and create a db schema or database

configure db properties like below

Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver

271 Configure Cluster Api

configure cluster api host and port details

customclusterapiurlhttplocalhost9343

bull ignore userpwd of cluster api properties

28 6 Metastore setup

281 Metastore Cassandra

In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo

bull Create tables and run insert scripts in Cassandra

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql

28 6 Metastore setup 9

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Contents

1 Introduction 311 What is Kafkawize 312 Background 313 Key features 414 41 glimpse 415 How is it developed 416 Architecture 417 Git Repositories 518 Developer 5

2 Getting Started 721 Demo 722 Prerequisites 723 1 Download Kafkawize 724 2 Download Metastore 725 3 License Key 826 4 Build KW Cluster Api Application 827 5 Configure Kafkawize UI Api Application 828 6 Metastore setup 929 7 Run KW and KWClusterApi 10210 8 Kafka Connectivity 11211 9 Final Check 11

3 Migration Kafkawize from 35 to 40 13

4 Features 1541 Security 1542 Teams and Role Management 1543 Login 1644 Dashboard 1645 Browse 1746 Requests 1847 Clusters 1948 Audit 2049 Approve Or Decline 22410 Users 23411 Teams 24

i

412 Synchronize Metadata 25413 Admin 26414 Profile 26415 Logout 26

5 Functionalities 27

6 Customize 29

7 Trouble shooting 31

8 Release Notes 33

9 Roadmap 35

10 Contact Me 37

11 Submit a Review 39

12 Indices and tables 41

ii

Kafkawize Documentation Release 41

Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization Read more

Contents

Contents 1

Kafkawize Documentation Release 41

2 Contents

CHAPTER 1

Introduction

11 What is Kafkawize

Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization

12 Background

Let me give some background Apache Kafka is widely being adapted in organizations irrespective of the scale Itrsquosunique features like scalability retention and reliability unlike the traditional messaging platforms makes it stand out

While adapting Kafka you would notice there are few manual activities like creating topics acls updating configu-ration etc There have been few topic management tools in the market which provide automation to certain extenthowever its not complete Assume there is a team requesting for a kafka topic with in an organization They wouldeither request for it through email with a template or any user interface The actual kafka team in the organization ex-ecutes the commands to create topics With Development Test Acceptance and Production environments managingthese configurations by the Kafka team and handling all the requests from various teams would become a hassle asyou see the growth of teams using Kafka It is just not manual activity but Audit time and effort the team has to putin

Apart from that this meta data (Kafka topics acls) which is stored in ZK could be easily stored in a good readableformat and can be used as source of truth

Hence I saw the need of Kafkawize ndash a self service topic management tool to keep ZERO interaction by other Teamswith Kafka team for topics acls or schemas creation by introducing roles and authorizations to users of teams It isdeveloped with a simple front end and backend with Spring boot It saves time and effort governs information avoidsrisk in kafka config loss avoids manual activities and mistakes maintains up to date SOT (source of truth)

3

Kafkawize Documentation Release 41

13 Key features

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

14 41 glimpse

15 How is it developed

Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it

16 Architecture

Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end

User Interface Api directly communicates between Frontend UI and Cluster API

Front end is built with AngularJs HTML and Java script

Cluster API acts as middle layer between Kafka brokers and UserInterface API

4 Chapter 1 Introduction

Kafkawize Documentation Release 41

Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client

Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth

Requests from users are directed to cluster api and also data is stored metastore

On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application

17 Git Repositories

UI Api httpsgithubcommuralibasanikafkawize

Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi

18 Developer

Muralidhar Basani

LinkedIn httpswwwlinkedincominmuralibasani

Web httpskafkawizecom

17 Git Repositories 5

Kafkawize Documentation Release 41

6 Chapter 1 Introduction

CHAPTER 2

Getting Started

21 Demo

Lets start with setting up Kafkawize on your WindowsLinuxUnix systems

22 Prerequisites

bull Any IDE like IntelliJNetbeans or Eclipse to build the applications

bull Java installed machines to run the applications

bull Rdbms (like OracleMySQL) or Apache Cassandra

23 1 Download Kafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi

OR

Download Kafkawize 41 bundle here Kafkawize_41

24 2 Download Metastore

If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload

If Rdbms is used as metastore download

Mysql httpsdevmysqlcomdownloads

7

Kafkawize Documentation Release 41

Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml

Or any other Rdbms is ok

25 3 License Key

License key is not required from 41 version

26 4 Build KW Cluster Api Application

This Api does the below

bull Receives and responds to calls from UI Api

bull Connects to Kafka Brokers with Kafka AdminClient Api

bull There is no connection to any metastore

bull Swagger enabled

Unzip kafkawizeclusterapizip and open in any IDE

Configure the application properties (srcmainresources) if port has to be changed else default port is 9343

and run maven command to create a runnable jar

mvn clean package

This should create a jar (kafkawizeclusterapi-41jar) in target dir

Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even

27 5 Configure Kafkawize UI Api Application

This Api does the below

bull Users interact with interface with this Api

bull All the end points in this application either connect to Metastore or Cluster Api or both

Unzip kafkawize-(version)zip and open in any IDE

Configure application properties

In kafkawizesrcmainresourcesapplicationproperties

default port is 9097 if port has to be changed else

serverport9097

If metastore is cassandra (from step 4)

setstore type as cassandra

customdbstoretype=cassandra

8 Chapter 2 Getting Started

Kafkawize Documentation Release 41

configure cassandra host port and keyspace

customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042

If metastore is rdbms (from step 4)

setstore type as rdbms

customdbstoretype=rdbms

bull Install and run Rdbms (like MysqlOracle) and create a db schema or database

configure db properties like below

Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver

271 Configure Cluster Api

configure cluster api host and port details

customclusterapiurlhttplocalhost9343

bull ignore userpwd of cluster api properties

28 6 Metastore setup

281 Metastore Cassandra

In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo

bull Create tables and run insert scripts in Cassandra

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql

28 6 Metastore setup 9

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

412 Synchronize Metadata 25413 Admin 26414 Profile 26415 Logout 26

5 Functionalities 27

6 Customize 29

7 Trouble shooting 31

8 Release Notes 33

9 Roadmap 35

10 Contact Me 37

11 Submit a Review 39

12 Indices and tables 41

ii

Kafkawize Documentation Release 41

Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization Read more

Contents

Contents 1

Kafkawize Documentation Release 41

2 Contents

CHAPTER 1

Introduction

11 What is Kafkawize

Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization

12 Background

Let me give some background Apache Kafka is widely being adapted in organizations irrespective of the scale Itrsquosunique features like scalability retention and reliability unlike the traditional messaging platforms makes it stand out

While adapting Kafka you would notice there are few manual activities like creating topics acls updating configu-ration etc There have been few topic management tools in the market which provide automation to certain extenthowever its not complete Assume there is a team requesting for a kafka topic with in an organization They wouldeither request for it through email with a template or any user interface The actual kafka team in the organization ex-ecutes the commands to create topics With Development Test Acceptance and Production environments managingthese configurations by the Kafka team and handling all the requests from various teams would become a hassle asyou see the growth of teams using Kafka It is just not manual activity but Audit time and effort the team has to putin

Apart from that this meta data (Kafka topics acls) which is stored in ZK could be easily stored in a good readableformat and can be used as source of truth

Hence I saw the need of Kafkawize ndash a self service topic management tool to keep ZERO interaction by other Teamswith Kafka team for topics acls or schemas creation by introducing roles and authorizations to users of teams It isdeveloped with a simple front end and backend with Spring boot It saves time and effort governs information avoidsrisk in kafka config loss avoids manual activities and mistakes maintains up to date SOT (source of truth)

3

Kafkawize Documentation Release 41

13 Key features

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

14 41 glimpse

15 How is it developed

Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it

16 Architecture

Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end

User Interface Api directly communicates between Frontend UI and Cluster API

Front end is built with AngularJs HTML and Java script

Cluster API acts as middle layer between Kafka brokers and UserInterface API

4 Chapter 1 Introduction

Kafkawize Documentation Release 41

Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client

Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth

Requests from users are directed to cluster api and also data is stored metastore

On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application

17 Git Repositories

UI Api httpsgithubcommuralibasanikafkawize

Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi

18 Developer

Muralidhar Basani

LinkedIn httpswwwlinkedincominmuralibasani

Web httpskafkawizecom

17 Git Repositories 5

Kafkawize Documentation Release 41

6 Chapter 1 Introduction

CHAPTER 2

Getting Started

21 Demo

Lets start with setting up Kafkawize on your WindowsLinuxUnix systems

22 Prerequisites

bull Any IDE like IntelliJNetbeans or Eclipse to build the applications

bull Java installed machines to run the applications

bull Rdbms (like OracleMySQL) or Apache Cassandra

23 1 Download Kafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi

OR

Download Kafkawize 41 bundle here Kafkawize_41

24 2 Download Metastore

If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload

If Rdbms is used as metastore download

Mysql httpsdevmysqlcomdownloads

7

Kafkawize Documentation Release 41

Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml

Or any other Rdbms is ok

25 3 License Key

License key is not required from 41 version

26 4 Build KW Cluster Api Application

This Api does the below

bull Receives and responds to calls from UI Api

bull Connects to Kafka Brokers with Kafka AdminClient Api

bull There is no connection to any metastore

bull Swagger enabled

Unzip kafkawizeclusterapizip and open in any IDE

Configure the application properties (srcmainresources) if port has to be changed else default port is 9343

and run maven command to create a runnable jar

mvn clean package

This should create a jar (kafkawizeclusterapi-41jar) in target dir

Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even

27 5 Configure Kafkawize UI Api Application

This Api does the below

bull Users interact with interface with this Api

bull All the end points in this application either connect to Metastore or Cluster Api or both

Unzip kafkawize-(version)zip and open in any IDE

Configure application properties

In kafkawizesrcmainresourcesapplicationproperties

default port is 9097 if port has to be changed else

serverport9097

If metastore is cassandra (from step 4)

setstore type as cassandra

customdbstoretype=cassandra

8 Chapter 2 Getting Started

Kafkawize Documentation Release 41

configure cassandra host port and keyspace

customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042

If metastore is rdbms (from step 4)

setstore type as rdbms

customdbstoretype=rdbms

bull Install and run Rdbms (like MysqlOracle) and create a db schema or database

configure db properties like below

Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver

271 Configure Cluster Api

configure cluster api host and port details

customclusterapiurlhttplocalhost9343

bull ignore userpwd of cluster api properties

28 6 Metastore setup

281 Metastore Cassandra

In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo

bull Create tables and run insert scripts in Cassandra

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql

28 6 Metastore setup 9

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization Read more

Contents

Contents 1

Kafkawize Documentation Release 41

2 Contents

CHAPTER 1

Introduction

11 What is Kafkawize

Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization

12 Background

Let me give some background Apache Kafka is widely being adapted in organizations irrespective of the scale Itrsquosunique features like scalability retention and reliability unlike the traditional messaging platforms makes it stand out

While adapting Kafka you would notice there are few manual activities like creating topics acls updating configu-ration etc There have been few topic management tools in the market which provide automation to certain extenthowever its not complete Assume there is a team requesting for a kafka topic with in an organization They wouldeither request for it through email with a template or any user interface The actual kafka team in the organization ex-ecutes the commands to create topics With Development Test Acceptance and Production environments managingthese configurations by the Kafka team and handling all the requests from various teams would become a hassle asyou see the growth of teams using Kafka It is just not manual activity but Audit time and effort the team has to putin

Apart from that this meta data (Kafka topics acls) which is stored in ZK could be easily stored in a good readableformat and can be used as source of truth

Hence I saw the need of Kafkawize ndash a self service topic management tool to keep ZERO interaction by other Teamswith Kafka team for topics acls or schemas creation by introducing roles and authorizations to users of teams It isdeveloped with a simple front end and backend with Spring boot It saves time and effort governs information avoidsrisk in kafka config loss avoids manual activities and mistakes maintains up to date SOT (source of truth)

3

Kafkawize Documentation Release 41

13 Key features

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

14 41 glimpse

15 How is it developed

Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it

16 Architecture

Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end

User Interface Api directly communicates between Frontend UI and Cluster API

Front end is built with AngularJs HTML and Java script

Cluster API acts as middle layer between Kafka brokers and UserInterface API

4 Chapter 1 Introduction

Kafkawize Documentation Release 41

Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client

Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth

Requests from users are directed to cluster api and also data is stored metastore

On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application

17 Git Repositories

UI Api httpsgithubcommuralibasanikafkawize

Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi

18 Developer

Muralidhar Basani

LinkedIn httpswwwlinkedincominmuralibasani

Web httpskafkawizecom

17 Git Repositories 5

Kafkawize Documentation Release 41

6 Chapter 1 Introduction

CHAPTER 2

Getting Started

21 Demo

Lets start with setting up Kafkawize on your WindowsLinuxUnix systems

22 Prerequisites

bull Any IDE like IntelliJNetbeans or Eclipse to build the applications

bull Java installed machines to run the applications

bull Rdbms (like OracleMySQL) or Apache Cassandra

23 1 Download Kafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi

OR

Download Kafkawize 41 bundle here Kafkawize_41

24 2 Download Metastore

If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload

If Rdbms is used as metastore download

Mysql httpsdevmysqlcomdownloads

7

Kafkawize Documentation Release 41

Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml

Or any other Rdbms is ok

25 3 License Key

License key is not required from 41 version

26 4 Build KW Cluster Api Application

This Api does the below

bull Receives and responds to calls from UI Api

bull Connects to Kafka Brokers with Kafka AdminClient Api

bull There is no connection to any metastore

bull Swagger enabled

Unzip kafkawizeclusterapizip and open in any IDE

Configure the application properties (srcmainresources) if port has to be changed else default port is 9343

and run maven command to create a runnable jar

mvn clean package

This should create a jar (kafkawizeclusterapi-41jar) in target dir

Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even

27 5 Configure Kafkawize UI Api Application

This Api does the below

bull Users interact with interface with this Api

bull All the end points in this application either connect to Metastore or Cluster Api or both

Unzip kafkawize-(version)zip and open in any IDE

Configure application properties

In kafkawizesrcmainresourcesapplicationproperties

default port is 9097 if port has to be changed else

serverport9097

If metastore is cassandra (from step 4)

setstore type as cassandra

customdbstoretype=cassandra

8 Chapter 2 Getting Started

Kafkawize Documentation Release 41

configure cassandra host port and keyspace

customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042

If metastore is rdbms (from step 4)

setstore type as rdbms

customdbstoretype=rdbms

bull Install and run Rdbms (like MysqlOracle) and create a db schema or database

configure db properties like below

Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver

271 Configure Cluster Api

configure cluster api host and port details

customclusterapiurlhttplocalhost9343

bull ignore userpwd of cluster api properties

28 6 Metastore setup

281 Metastore Cassandra

In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo

bull Create tables and run insert scripts in Cassandra

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql

28 6 Metastore setup 9

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

2 Contents

CHAPTER 1

Introduction

11 What is Kafkawize

Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization

12 Background

Let me give some background Apache Kafka is widely being adapted in organizations irrespective of the scale Itrsquosunique features like scalability retention and reliability unlike the traditional messaging platforms makes it stand out

While adapting Kafka you would notice there are few manual activities like creating topics acls updating configu-ration etc There have been few topic management tools in the market which provide automation to certain extenthowever its not complete Assume there is a team requesting for a kafka topic with in an organization They wouldeither request for it through email with a template or any user interface The actual kafka team in the organization ex-ecutes the commands to create topics With Development Test Acceptance and Production environments managingthese configurations by the Kafka team and handling all the requests from various teams would become a hassle asyou see the growth of teams using Kafka It is just not manual activity but Audit time and effort the team has to putin

Apart from that this meta data (Kafka topics acls) which is stored in ZK could be easily stored in a good readableformat and can be used as source of truth

Hence I saw the need of Kafkawize ndash a self service topic management tool to keep ZERO interaction by other Teamswith Kafka team for topics acls or schemas creation by introducing roles and authorizations to users of teams It isdeveloped with a simple front end and backend with Spring boot It saves time and effort governs information avoidsrisk in kafka config loss avoids manual activities and mistakes maintains up to date SOT (source of truth)

3

Kafkawize Documentation Release 41

13 Key features

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

14 41 glimpse

15 How is it developed

Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it

16 Architecture

Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end

User Interface Api directly communicates between Frontend UI and Cluster API

Front end is built with AngularJs HTML and Java script

Cluster API acts as middle layer between Kafka brokers and UserInterface API

4 Chapter 1 Introduction

Kafkawize Documentation Release 41

Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client

Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth

Requests from users are directed to cluster api and also data is stored metastore

On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application

17 Git Repositories

UI Api httpsgithubcommuralibasanikafkawize

Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi

18 Developer

Muralidhar Basani

LinkedIn httpswwwlinkedincominmuralibasani

Web httpskafkawizecom

17 Git Repositories 5

Kafkawize Documentation Release 41

6 Chapter 1 Introduction

CHAPTER 2

Getting Started

21 Demo

Lets start with setting up Kafkawize on your WindowsLinuxUnix systems

22 Prerequisites

bull Any IDE like IntelliJNetbeans or Eclipse to build the applications

bull Java installed machines to run the applications

bull Rdbms (like OracleMySQL) or Apache Cassandra

23 1 Download Kafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi

OR

Download Kafkawize 41 bundle here Kafkawize_41

24 2 Download Metastore

If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload

If Rdbms is used as metastore download

Mysql httpsdevmysqlcomdownloads

7

Kafkawize Documentation Release 41

Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml

Or any other Rdbms is ok

25 3 License Key

License key is not required from 41 version

26 4 Build KW Cluster Api Application

This Api does the below

bull Receives and responds to calls from UI Api

bull Connects to Kafka Brokers with Kafka AdminClient Api

bull There is no connection to any metastore

bull Swagger enabled

Unzip kafkawizeclusterapizip and open in any IDE

Configure the application properties (srcmainresources) if port has to be changed else default port is 9343

and run maven command to create a runnable jar

mvn clean package

This should create a jar (kafkawizeclusterapi-41jar) in target dir

Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even

27 5 Configure Kafkawize UI Api Application

This Api does the below

bull Users interact with interface with this Api

bull All the end points in this application either connect to Metastore or Cluster Api or both

Unzip kafkawize-(version)zip and open in any IDE

Configure application properties

In kafkawizesrcmainresourcesapplicationproperties

default port is 9097 if port has to be changed else

serverport9097

If metastore is cassandra (from step 4)

setstore type as cassandra

customdbstoretype=cassandra

8 Chapter 2 Getting Started

Kafkawize Documentation Release 41

configure cassandra host port and keyspace

customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042

If metastore is rdbms (from step 4)

setstore type as rdbms

customdbstoretype=rdbms

bull Install and run Rdbms (like MysqlOracle) and create a db schema or database

configure db properties like below

Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver

271 Configure Cluster Api

configure cluster api host and port details

customclusterapiurlhttplocalhost9343

bull ignore userpwd of cluster api properties

28 6 Metastore setup

281 Metastore Cassandra

In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo

bull Create tables and run insert scripts in Cassandra

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql

28 6 Metastore setup 9

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

CHAPTER 1

Introduction

11 What is Kafkawize

Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization

12 Background

Let me give some background Apache Kafka is widely being adapted in organizations irrespective of the scale Itrsquosunique features like scalability retention and reliability unlike the traditional messaging platforms makes it stand out

While adapting Kafka you would notice there are few manual activities like creating topics acls updating configu-ration etc There have been few topic management tools in the market which provide automation to certain extenthowever its not complete Assume there is a team requesting for a kafka topic with in an organization They wouldeither request for it through email with a template or any user interface The actual kafka team in the organization ex-ecutes the commands to create topics With Development Test Acceptance and Production environments managingthese configurations by the Kafka team and handling all the requests from various teams would become a hassle asyou see the growth of teams using Kafka It is just not manual activity but Audit time and effort the team has to putin

Apart from that this meta data (Kafka topics acls) which is stored in ZK could be easily stored in a good readableformat and can be used as source of truth

Hence I saw the need of Kafkawize ndash a self service topic management tool to keep ZERO interaction by other Teamswith Kafka team for topics acls or schemas creation by introducing roles and authorizations to users of teams It isdeveloped with a simple front end and backend with Spring boot It saves time and effort governs information avoidsrisk in kafka config loss avoids manual activities and mistakes maintains up to date SOT (source of truth)

3

Kafkawize Documentation Release 41

13 Key features

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

14 41 glimpse

15 How is it developed

Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it

16 Architecture

Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end

User Interface Api directly communicates between Frontend UI and Cluster API

Front end is built with AngularJs HTML and Java script

Cluster API acts as middle layer between Kafka brokers and UserInterface API

4 Chapter 1 Introduction

Kafkawize Documentation Release 41

Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client

Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth

Requests from users are directed to cluster api and also data is stored metastore

On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application

17 Git Repositories

UI Api httpsgithubcommuralibasanikafkawize

Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi

18 Developer

Muralidhar Basani

LinkedIn httpswwwlinkedincominmuralibasani

Web httpskafkawizecom

17 Git Repositories 5

Kafkawize Documentation Release 41

6 Chapter 1 Introduction

CHAPTER 2

Getting Started

21 Demo

Lets start with setting up Kafkawize on your WindowsLinuxUnix systems

22 Prerequisites

bull Any IDE like IntelliJNetbeans or Eclipse to build the applications

bull Java installed machines to run the applications

bull Rdbms (like OracleMySQL) or Apache Cassandra

23 1 Download Kafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi

OR

Download Kafkawize 41 bundle here Kafkawize_41

24 2 Download Metastore

If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload

If Rdbms is used as metastore download

Mysql httpsdevmysqlcomdownloads

7

Kafkawize Documentation Release 41

Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml

Or any other Rdbms is ok

25 3 License Key

License key is not required from 41 version

26 4 Build KW Cluster Api Application

This Api does the below

bull Receives and responds to calls from UI Api

bull Connects to Kafka Brokers with Kafka AdminClient Api

bull There is no connection to any metastore

bull Swagger enabled

Unzip kafkawizeclusterapizip and open in any IDE

Configure the application properties (srcmainresources) if port has to be changed else default port is 9343

and run maven command to create a runnable jar

mvn clean package

This should create a jar (kafkawizeclusterapi-41jar) in target dir

Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even

27 5 Configure Kafkawize UI Api Application

This Api does the below

bull Users interact with interface with this Api

bull All the end points in this application either connect to Metastore or Cluster Api or both

Unzip kafkawize-(version)zip and open in any IDE

Configure application properties

In kafkawizesrcmainresourcesapplicationproperties

default port is 9097 if port has to be changed else

serverport9097

If metastore is cassandra (from step 4)

setstore type as cassandra

customdbstoretype=cassandra

8 Chapter 2 Getting Started

Kafkawize Documentation Release 41

configure cassandra host port and keyspace

customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042

If metastore is rdbms (from step 4)

setstore type as rdbms

customdbstoretype=rdbms

bull Install and run Rdbms (like MysqlOracle) and create a db schema or database

configure db properties like below

Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver

271 Configure Cluster Api

configure cluster api host and port details

customclusterapiurlhttplocalhost9343

bull ignore userpwd of cluster api properties

28 6 Metastore setup

281 Metastore Cassandra

In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo

bull Create tables and run insert scripts in Cassandra

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql

28 6 Metastore setup 9

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

13 Key features

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

14 41 glimpse

15 How is it developed

Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it

16 Architecture

Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end

User Interface Api directly communicates between Frontend UI and Cluster API

Front end is built with AngularJs HTML and Java script

Cluster API acts as middle layer between Kafka brokers and UserInterface API

4 Chapter 1 Introduction

Kafkawize Documentation Release 41

Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client

Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth

Requests from users are directed to cluster api and also data is stored metastore

On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application

17 Git Repositories

UI Api httpsgithubcommuralibasanikafkawize

Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi

18 Developer

Muralidhar Basani

LinkedIn httpswwwlinkedincominmuralibasani

Web httpskafkawizecom

17 Git Repositories 5

Kafkawize Documentation Release 41

6 Chapter 1 Introduction

CHAPTER 2

Getting Started

21 Demo

Lets start with setting up Kafkawize on your WindowsLinuxUnix systems

22 Prerequisites

bull Any IDE like IntelliJNetbeans or Eclipse to build the applications

bull Java installed machines to run the applications

bull Rdbms (like OracleMySQL) or Apache Cassandra

23 1 Download Kafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi

OR

Download Kafkawize 41 bundle here Kafkawize_41

24 2 Download Metastore

If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload

If Rdbms is used as metastore download

Mysql httpsdevmysqlcomdownloads

7

Kafkawize Documentation Release 41

Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml

Or any other Rdbms is ok

25 3 License Key

License key is not required from 41 version

26 4 Build KW Cluster Api Application

This Api does the below

bull Receives and responds to calls from UI Api

bull Connects to Kafka Brokers with Kafka AdminClient Api

bull There is no connection to any metastore

bull Swagger enabled

Unzip kafkawizeclusterapizip and open in any IDE

Configure the application properties (srcmainresources) if port has to be changed else default port is 9343

and run maven command to create a runnable jar

mvn clean package

This should create a jar (kafkawizeclusterapi-41jar) in target dir

Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even

27 5 Configure Kafkawize UI Api Application

This Api does the below

bull Users interact with interface with this Api

bull All the end points in this application either connect to Metastore or Cluster Api or both

Unzip kafkawize-(version)zip and open in any IDE

Configure application properties

In kafkawizesrcmainresourcesapplicationproperties

default port is 9097 if port has to be changed else

serverport9097

If metastore is cassandra (from step 4)

setstore type as cassandra

customdbstoretype=cassandra

8 Chapter 2 Getting Started

Kafkawize Documentation Release 41

configure cassandra host port and keyspace

customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042

If metastore is rdbms (from step 4)

setstore type as rdbms

customdbstoretype=rdbms

bull Install and run Rdbms (like MysqlOracle) and create a db schema or database

configure db properties like below

Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver

271 Configure Cluster Api

configure cluster api host and port details

customclusterapiurlhttplocalhost9343

bull ignore userpwd of cluster api properties

28 6 Metastore setup

281 Metastore Cassandra

In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo

bull Create tables and run insert scripts in Cassandra

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql

28 6 Metastore setup 9

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client

Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth

Requests from users are directed to cluster api and also data is stored metastore

On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application

17 Git Repositories

UI Api httpsgithubcommuralibasanikafkawize

Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi

18 Developer

Muralidhar Basani

LinkedIn httpswwwlinkedincominmuralibasani

Web httpskafkawizecom

17 Git Repositories 5

Kafkawize Documentation Release 41

6 Chapter 1 Introduction

CHAPTER 2

Getting Started

21 Demo

Lets start with setting up Kafkawize on your WindowsLinuxUnix systems

22 Prerequisites

bull Any IDE like IntelliJNetbeans or Eclipse to build the applications

bull Java installed machines to run the applications

bull Rdbms (like OracleMySQL) or Apache Cassandra

23 1 Download Kafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi

OR

Download Kafkawize 41 bundle here Kafkawize_41

24 2 Download Metastore

If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload

If Rdbms is used as metastore download

Mysql httpsdevmysqlcomdownloads

7

Kafkawize Documentation Release 41

Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml

Or any other Rdbms is ok

25 3 License Key

License key is not required from 41 version

26 4 Build KW Cluster Api Application

This Api does the below

bull Receives and responds to calls from UI Api

bull Connects to Kafka Brokers with Kafka AdminClient Api

bull There is no connection to any metastore

bull Swagger enabled

Unzip kafkawizeclusterapizip and open in any IDE

Configure the application properties (srcmainresources) if port has to be changed else default port is 9343

and run maven command to create a runnable jar

mvn clean package

This should create a jar (kafkawizeclusterapi-41jar) in target dir

Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even

27 5 Configure Kafkawize UI Api Application

This Api does the below

bull Users interact with interface with this Api

bull All the end points in this application either connect to Metastore or Cluster Api or both

Unzip kafkawize-(version)zip and open in any IDE

Configure application properties

In kafkawizesrcmainresourcesapplicationproperties

default port is 9097 if port has to be changed else

serverport9097

If metastore is cassandra (from step 4)

setstore type as cassandra

customdbstoretype=cassandra

8 Chapter 2 Getting Started

Kafkawize Documentation Release 41

configure cassandra host port and keyspace

customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042

If metastore is rdbms (from step 4)

setstore type as rdbms

customdbstoretype=rdbms

bull Install and run Rdbms (like MysqlOracle) and create a db schema or database

configure db properties like below

Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver

271 Configure Cluster Api

configure cluster api host and port details

customclusterapiurlhttplocalhost9343

bull ignore userpwd of cluster api properties

28 6 Metastore setup

281 Metastore Cassandra

In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo

bull Create tables and run insert scripts in Cassandra

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql

28 6 Metastore setup 9

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

6 Chapter 1 Introduction

CHAPTER 2

Getting Started

21 Demo

Lets start with setting up Kafkawize on your WindowsLinuxUnix systems

22 Prerequisites

bull Any IDE like IntelliJNetbeans or Eclipse to build the applications

bull Java installed machines to run the applications

bull Rdbms (like OracleMySQL) or Apache Cassandra

23 1 Download Kafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi

OR

Download Kafkawize 41 bundle here Kafkawize_41

24 2 Download Metastore

If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload

If Rdbms is used as metastore download

Mysql httpsdevmysqlcomdownloads

7

Kafkawize Documentation Release 41

Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml

Or any other Rdbms is ok

25 3 License Key

License key is not required from 41 version

26 4 Build KW Cluster Api Application

This Api does the below

bull Receives and responds to calls from UI Api

bull Connects to Kafka Brokers with Kafka AdminClient Api

bull There is no connection to any metastore

bull Swagger enabled

Unzip kafkawizeclusterapizip and open in any IDE

Configure the application properties (srcmainresources) if port has to be changed else default port is 9343

and run maven command to create a runnable jar

mvn clean package

This should create a jar (kafkawizeclusterapi-41jar) in target dir

Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even

27 5 Configure Kafkawize UI Api Application

This Api does the below

bull Users interact with interface with this Api

bull All the end points in this application either connect to Metastore or Cluster Api or both

Unzip kafkawize-(version)zip and open in any IDE

Configure application properties

In kafkawizesrcmainresourcesapplicationproperties

default port is 9097 if port has to be changed else

serverport9097

If metastore is cassandra (from step 4)

setstore type as cassandra

customdbstoretype=cassandra

8 Chapter 2 Getting Started

Kafkawize Documentation Release 41

configure cassandra host port and keyspace

customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042

If metastore is rdbms (from step 4)

setstore type as rdbms

customdbstoretype=rdbms

bull Install and run Rdbms (like MysqlOracle) and create a db schema or database

configure db properties like below

Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver

271 Configure Cluster Api

configure cluster api host and port details

customclusterapiurlhttplocalhost9343

bull ignore userpwd of cluster api properties

28 6 Metastore setup

281 Metastore Cassandra

In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo

bull Create tables and run insert scripts in Cassandra

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql

28 6 Metastore setup 9

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

CHAPTER 2

Getting Started

21 Demo

Lets start with setting up Kafkawize on your WindowsLinuxUnix systems

22 Prerequisites

bull Any IDE like IntelliJNetbeans or Eclipse to build the applications

bull Java installed machines to run the applications

bull Rdbms (like OracleMySQL) or Apache Cassandra

23 1 Download Kafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize

Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi

OR

Download Kafkawize 41 bundle here Kafkawize_41

24 2 Download Metastore

If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload

If Rdbms is used as metastore download

Mysql httpsdevmysqlcomdownloads

7

Kafkawize Documentation Release 41

Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml

Or any other Rdbms is ok

25 3 License Key

License key is not required from 41 version

26 4 Build KW Cluster Api Application

This Api does the below

bull Receives and responds to calls from UI Api

bull Connects to Kafka Brokers with Kafka AdminClient Api

bull There is no connection to any metastore

bull Swagger enabled

Unzip kafkawizeclusterapizip and open in any IDE

Configure the application properties (srcmainresources) if port has to be changed else default port is 9343

and run maven command to create a runnable jar

mvn clean package

This should create a jar (kafkawizeclusterapi-41jar) in target dir

Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even

27 5 Configure Kafkawize UI Api Application

This Api does the below

bull Users interact with interface with this Api

bull All the end points in this application either connect to Metastore or Cluster Api or both

Unzip kafkawize-(version)zip and open in any IDE

Configure application properties

In kafkawizesrcmainresourcesapplicationproperties

default port is 9097 if port has to be changed else

serverport9097

If metastore is cassandra (from step 4)

setstore type as cassandra

customdbstoretype=cassandra

8 Chapter 2 Getting Started

Kafkawize Documentation Release 41

configure cassandra host port and keyspace

customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042

If metastore is rdbms (from step 4)

setstore type as rdbms

customdbstoretype=rdbms

bull Install and run Rdbms (like MysqlOracle) and create a db schema or database

configure db properties like below

Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver

271 Configure Cluster Api

configure cluster api host and port details

customclusterapiurlhttplocalhost9343

bull ignore userpwd of cluster api properties

28 6 Metastore setup

281 Metastore Cassandra

In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo

bull Create tables and run insert scripts in Cassandra

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql

28 6 Metastore setup 9

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml

Or any other Rdbms is ok

25 3 License Key

License key is not required from 41 version

26 4 Build KW Cluster Api Application

This Api does the below

bull Receives and responds to calls from UI Api

bull Connects to Kafka Brokers with Kafka AdminClient Api

bull There is no connection to any metastore

bull Swagger enabled

Unzip kafkawizeclusterapizip and open in any IDE

Configure the application properties (srcmainresources) if port has to be changed else default port is 9343

and run maven command to create a runnable jar

mvn clean package

This should create a jar (kafkawizeclusterapi-41jar) in target dir

Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even

27 5 Configure Kafkawize UI Api Application

This Api does the below

bull Users interact with interface with this Api

bull All the end points in this application either connect to Metastore or Cluster Api or both

Unzip kafkawize-(version)zip and open in any IDE

Configure application properties

In kafkawizesrcmainresourcesapplicationproperties

default port is 9097 if port has to be changed else

serverport9097

If metastore is cassandra (from step 4)

setstore type as cassandra

customdbstoretype=cassandra

8 Chapter 2 Getting Started

Kafkawize Documentation Release 41

configure cassandra host port and keyspace

customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042

If metastore is rdbms (from step 4)

setstore type as rdbms

customdbstoretype=rdbms

bull Install and run Rdbms (like MysqlOracle) and create a db schema or database

configure db properties like below

Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver

271 Configure Cluster Api

configure cluster api host and port details

customclusterapiurlhttplocalhost9343

bull ignore userpwd of cluster api properties

28 6 Metastore setup

281 Metastore Cassandra

In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo

bull Create tables and run insert scripts in Cassandra

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql

28 6 Metastore setup 9

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

configure cassandra host port and keyspace

customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042

If metastore is rdbms (from step 4)

setstore type as rdbms

customdbstoretype=rdbms

bull Install and run Rdbms (like MysqlOracle) and create a db schema or database

configure db properties like below

Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver

271 Configure Cluster Api

configure cluster api host and port details

customclusterapiurlhttplocalhost9343

bull ignore userpwd of cluster api properties

28 6 Metastore setup

281 Metastore Cassandra

In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo

bull Create tables and run insert scripts in Cassandra

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql

28 6 Metastore setup 9

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

282 Metastore Rdbms

In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted

Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto

Configure in application properties

customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true

If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically

If the customdbscriptsexecution property is set to manual all the scripts should be executed manually

bull Install and run MysqlOracle and create a db schema or database

bull Create tables and run insert scripts in Database

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql

kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql

(Scripts available in kafkawize-41zip)

bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI

Run maven command to create a runnable jar

mvn clean package

This should create a jar in target dir (kafkawizetargetkafkawize-41jar)

29 7 Run KW and KWClusterApi

Run

java -jar kafkawizeclusterapi-41jar

Run

java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar

If application is running you can access UI from http[host][port]kafkawize

10 Chapter 2 Getting Started

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

210 8 Kafka Connectivity

Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed

bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)

Examples SSL Based Acl (Note of double quotes in the below command if copied properly)

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster

Examples IP Based Acl

binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster

bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application

211 9 Final Check

bull Cluster Api is running

bull Metastore (Cassandra or Rdbms) is running and has tables and data

bull UI Api is running

bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)

bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize

Default users passwords and roles

uiuser1user USER

uiuser2user USER

uiuser3user USER

uiuser4user ADMIN

uiuser5user ADMIN

superuseruser SUPERUSER

210 8 Kafka Connectivity 11

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

12 Chapter 2 Getting Started

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

CHAPTER 3

Migration Kafkawize from 35 to 40

1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties

customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false

4 Make sure to update the other required config for application properties

customkafkawizeversion=lt41gt copy from the new downloaded zip

13

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

14 Chapter 3 Migration Kafkawize from 35 to 40

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

CHAPTER 4

Features

Kafkawize is built with the key features as below

- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms

41 Security

There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application

On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it

Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues

42 Teams and Role Management

Kafkawize comes with 3 mains roles

- USER- ADMIN- SUPERUSER

lsquoUserrsquo has the rights to request and browse data

15

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

lsquoAdminrsquo has the all the rights as USER and approve all requests

lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl

A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to

43 Login

Portal can be accessed only if the user logs in New users are created by Super users

After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions

Default login url httplocalhost9097kafkawize

44 Dashboard

After a user is logged in heshe will be redirected to homepage

16 Chapter 4 Features

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

45 Browse

All the logged in users with any roles can browse the below components

451 Topics

A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application

A filter option is also provided and users can search for specific topics

Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)

452 Acls

Acl - Access Controls

From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic

Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics

45 Browse 17

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen

Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user

There can be multiple producers and multiple consumers for a topic owning by different teams

Unknown team is displayed in case of that acl is not owned by any team

46 Requests

All the users can request for topics acls or avro schemas

Topic Requests can be approved by Admins or Super users from the same team

Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests

461 Topics

To request for a topic all the mandatory parameters need to be provided

Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required

Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page

462 Acls

To request for a acl all the mandatory parameters need to be provided

18 Chapter 4 Features

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

After a user selects an environment topics are loaded and topic has to be selected

Acl type Producer or Consumer

IP Addresses or SSL DN name should be provided for authorizations

Only one IP Address or one SSL should be filled

On submission you can view your requests in AuditMy Requests and it can be deleted if required

463 Avro Schemas

An Avro schema can also be requested for a particular subject (topic)

47 Clusters

All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo

47 Clusters 19

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added

A cluster can be deleted by any Admin user or super user

48 Audit

All the requests from users are audited and can be viewed

481 My Topic Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

20 Chapter 4 Features

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

482 My Acl Requests

Requests (TopicsAclsSchemas) from their own team can be viewed and deleted

width 500px align center

483 Activity Log

All the requests requested and approved can be viewed Users loginlogout are not part of this log yet

48 Audit 21

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

49 Approve Or Decline

Topic Requests can be approved or declined by Admins or Super users from the same team

Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team

Schema Requests can be approved by Admins or Super users from the same team

Notifications are visible on the top right side of the portal

491 Topics

If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role

492 Acls

If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role

22 Chapter 4 Features

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

493 Avro Schemas

If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role

410 Users

All the users are visible to any logged in user A new user can be addeddeleted by only Super users

4101 View Users

From this page all the users can be seen and also be deleted

4102 Add User

With this form a new user can be added

410 Users 23

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

411 Teams

All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers

4111 View Teams

From this page all the users can be seen

4112 Add Team

With this form a new team can be added

24 Chapter 4 Features

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

412 Synchronize Metadata

A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners

This feature is possible with Synchronization of Topics or Acls

4121 Topics

After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team

4122 Acls

After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl

412 Synchronize Metadata 25

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

413 Admin

4131 Server Config

All the Server configuration including custom properties (application properties can be viewed)

414 Profile

All Users can view their profile with the button available in the top right corner of the portal

4141 Change Password

All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon

415 Logout

Users can logout after clicking on the logout button on the top right corner Session will be killed after this action

26 Chapter 4 Features

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

CHAPTER 5

Functionalities

bull Create Topics on Kafka Clusters Automated (Request and Approve)

bull Create Acls Automated (Request and Approve)

bull Register Avro schemas Automated (Request and Approve)

bull Search Topics with a User Interface

bull Search Acls with a User Interfaces

bull Search Producer and Consumer Teams

bull Synchronize Kafka topics with Teams

bull Synchronize Kafka acls with Teams

bull Users Teams and Roles based authorizations on features

bull View server configuration (application properties)

bull Audit of activities performed by users

bull Login and Logout

27

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

28 Chapter 5 Functionalities

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

CHAPTER 6

Customize

Kafkawize can be customized according to your requirement very easily

As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)

29

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

30 Chapter 6 Customize

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

CHAPTER 7

Trouble shooting

bull Check for logs in Cluster Api UI Api

bull Check if user has the right access

bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize

31

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

32 Chapter 7 Trouble shooting

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

CHAPTER 8

Release Notes

Kafkawize 41 release notes

Changes include 1 Critical bug fix - concurrent user access 2 Removed license check

33

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

34 Chapter 8 Release Notes

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

CHAPTER 9

Roadmap

bull Topic promotion

bull Email notifications

bull Delete Topic

bull Delete Acl

35

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

36 Chapter 9 Roadmap

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

CHAPTER 10

Contact Me

If you have any issues with kafkawize please raise it in

httpsgithubcommuralibasanikafkawizeissues

If you want to post a topic or question either you can post it in Stackoverflow or

httpskafkawizecomcommunity

You can also send me an email at infokafkawizecom or kafkawizegmailcom

You can also use the form httpskafkawizecomcontact-me

Muralidhar Basani

Linkedin httpswwwlinkedincominmuralibasani

37

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

38 Chapter 10 Contact Me

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

CHAPTER 11

Submit a Review

I will be pleased if you can write a testimonial about kafkawize at

httpskafkawizecomkafkawize_reviews

Please do not forget to star my project

httpsgithubcommuralibasanikafkawizestargazers

Thank you

39

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

Kafkawize Documentation Release 41

40 Chapter 11 Submit a Review

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables

CHAPTER 12

Indices and tables

bull genindex

bull modindex

bull search

41

  • Introduction
    • What is Kafkawize
    • Background
    • Key features
    • 41 glimpse
    • How is it developed
    • Architecture
    • Git Repositories
    • Developer
      • Getting Started
        • Demo
        • Prerequisites
        • 1 Download Kafkawize
        • 2 Download Metastore
        • 3 License Key
        • 4 Build KW Cluster Api Application
        • 5 Configure Kafkawize UI Api Application
        • 6 Metastore setup
        • 7 Run KW and KWClusterApi
        • 8 Kafka Connectivity
        • 9 Final Check
          • Migration Kafkawize from 35 to 40
          • Features
            • Security
            • Teams and Role Management
            • Login
            • Dashboard
            • Browse
            • Requests
            • Clusters
            • Audit
            • Approve Or Decline
            • Users
            • Teams
            • Synchronize Metadata
            • Admin
            • Profile
            • Logout
              • Functionalities
              • Customize
              • Trouble shooting
              • Release Notes
              • Roadmap
              • Contact Me
              • Submit a Review
              • Indices and tables