Download Kafka Mac

Posted on
Mac
Contents
  • Install Apache Kafka on Mac

    • To install Apache Kafka on Mac, Java is the only prerequisite. First we shall look into the installation steps of Java and then we shall setup Apache Kafka and run it on the Mac.

    Let us continue with the following steps to install Kafka on your machine. Step 3.1 - Download Kafka. To install Kafka on your machine, click on the below link −. I have started following the apache kafka documentation, but later thought to find the homebrew installer of Kafka. So to install Kafka on Mac using Homebrew: Now we will initialize the Kafka.

    Install Java

    1. Open a browser and hit the url [http://www.oracle.com/technetwork/java/javase/downloads/index.html].
    2. Click on JDK, check “Accept License Agreement” and download .dmg file for installation on Mac.
    3. Double click on the downloaded file and proceed with the installation steps.
    4. You have successfully installed Java. You may delete the .dmg file, as memory is costly on Mac.

    You may verify the installation of Java on Mac, by running the following command on a Terminal.

    java -version

    Install Apache Kafka on Mac

    KafkaDownload kafka manager
    1. Download the latest Apache Kafka from [https://kafka.apache.org/downloads] under Binary downloads.
    2. Click on any of the binary downloads, or choose a specific scala version if you have any dependency with scala in your development.
    3. Go with the recommended mirror site.
    4. Extract the contents. Navigate to root of Apache Kafka folder and open a Terminal. Or Open a Terminal and navigate to the root directory of Apache Kafka.
      apples-MacBook-Pro:kafka_2.12-1.0.0Prasanth$ ls
    Tool

    Start Zookeeper

    Apache Kafka depends on Zookeeper for cluster management. Hence, prior to starting Kafka, Zookeeper has to be started. There is no need to explicitly install Zookeeper, as it comes included with Apache Kafka.

    From the root of Apache Kafka, run the following command to start Zookeeper :

    ~$sh bin/zookeeper-server-start.sh config/zookeeper.properties

    The zookeeper should be started with a similar following trace in the output.

    apples-MacBook-Pro:kafka_2.12-1.0.0Prasanth$ sh bin/zookeeper-server-start.sh config/zookeeper.properties
    [2017-12-3115:18:36,556] INFO Reading configuration from:config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
    [2017-12-3115:18:36,560] INFO autopurge.snapRetainCount set to3 (org.apache.zookeeper.server.DatadirCleanupManager)
    [2017-12-3115:18:36,560] INFO autopurge.purgeInterval set to0 (org.apache.zookeeper.server.DatadirCleanupManager)
    [2017-12-3115:18:36,560] INFO Purge task isnotscheduled. (org.apache.zookeeper.server.DatadirCleanupManager)
    [2017-12-3115:18:36,560] WARN Either no config orno quorum defined inconfig, running instandalone mode (org.apache.zookeeper.server.quorum.QuorumPeerMain)
    [2017-12-3115:18:36,582] INFO Reading configuration from:config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
    [2017-12-3115:18:36,582] INFO Starting server (org.apache.zookeeper.server.ZooKeeperServerMain)
    [2017-12-3115:18:36,650] INFO Server environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on03/23/201710:13GMT (org.apache.zookeeper.server.ZooKeeperServer)
    [2017-12-3115:18:36,650] INFO Server environment:host.name=192.168.0.104 (org.apache.zookeeper.server.ZooKeeperServer)

    Start Apache Kafka Server

    Open another Terminal and run the following command from the root of Apache Kafka to start Apache Kafka.

    ~$sh bin/kafka-server-start.sh config/server.properties

    Following should be the end of trace stating that Kafka server is started.

    [2017-12-3115:52:20,291] INFO Initiating client connection, connectString=localhost:2181sessionTimeout=6000watcher=org.I0Itec.zkclient.ZkClient@4690b489 (org.apache.zookeeper.ZooKeeper)
    [2017-12-3115:52:20,307] INFO Waiting forkeeper state SyncConnected (org.I0Itec.zkclient.ZkClient)
    [2017-12-3115:52:20,310] INFO Opening socket connection toserver localhost/0:0:0:0:0:0:0:1:2181.Will notattempt toauthenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
    [2017-12-3115:52:20,333] INFO Socket connection established tolocalhost/0:0:0:0:0:0:0:1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
    [2017-12-3115:52:20,423] INFO Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid=0x160abf917fa0000, negotiated timeout=6000 (org.apache.zookeeper.ClientCnxn)
    [2017-12-3115:52:20,425] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
    [2017-12-3115:52:20,798] INFO Cluster ID=diRCJM2mQ4qI9KzjiiLRVA (kafka.server.KafkaServer)
    [2017-12-3115:52:20,820] WARN No meta.properties file under dir/tmp/kafka-logs/meta.properties (kafka.server.BrokerMetadataCheckpoint)
    [2017-12-3115:52:20,866] INFO [ThrottledRequestReaper-Fetch]:Starting (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
    [2017-12-3115:52:20,869] INFO [ThrottledRequestReaper-Produce]:Starting (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
    [2017-12-3115:52:20,870] INFO [ThrottledRequestReaper-Request]:Starting (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
    [2017-12-3115:52:20,923] INFO Log directory'/tmp/kafka-logs'notfound, creating it. (kafka.log.LogManager)
    [2017-12-3115:52:20,943] INFO Loading logs. (kafka.log.LogManager)
    [2017-12-3115:52:20,973] INFO Logs loading complete in29ms. (kafka.log.LogManager)
    [2017-12-3115:52:21,155] INFO Starting log cleanup withaperiod of300000ms. (kafka.log.LogManager)
    [2017-12-3115:52:21,159] INFO Starting log flusher withadefaultperiod of9223372036854775807ms. (kafka.log.LogManager)
    [2017-12-3115:52:21,535] INFO Awaiting socket connections on0.0.0.0:9092. (kafka.network.Acceptor)
    [2017-12-3115:52:21,538] INFO [SocketServer brokerId=0] Started1acceptor threads (kafka.network.SocketServer)
    [2017-12-3115:52:21,571] INFO [ExpirationReaper-0-Produce]:Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
    [2017-12-3115:52:21,573] INFO [ExpirationReaper-0-Fetch]:Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
    [2017-12-3115:52:21,575] INFO [ExpirationReaper-0-DeleteRecords]:Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
    [2017-12-3115:52:21,588] INFO [LogDirFailureHandler]:Starting (kafka.server.ReplicaManager$LogDirFailureHandler)
    [2017-12-3115:52:21,653] INFO [ExpirationReaper-0-topic]:Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
    [2017-12-3115:52:21,658] INFO [ExpirationReaper-0-Heartbeat]:Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
    [2017-12-3115:52:21,658] INFO Creating/controller (isit secure? false) (kafka.utils.ZKCheckedEphemeral)
    [2017-12-3115:52:21,659] INFO [ExpirationReaper-0-Rebalance]:Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
    [2017-12-3115:52:21,666] INFO Result of znode creation is:OK (kafka.utils.ZKCheckedEphemeral)
    [2017-12-3115:52:21,683] INFO [GroupCoordinator0]:Starting up. (kafka.coordinator.group.GroupCoordinator)
    [2017-12-3115:52:21,684] INFO [GroupCoordinator0]:Startup complete. (kafka.coordinator.group.GroupCoordinator)
    [2017-12-3115:52:21,688] INFO [GroupMetadataManager brokerId=0] Removed0expired offsets in4milliseconds. (kafka.coordinator.group.GroupMetadataManager)
    [2017-12-3115:52:21,727] INFO [ProducerId Manager0]:Acquired newproducerId block (brokerId:0,blockStartProducerId:0,blockEndProducerId:999) by writing toZk with path version1 (kafka.coordinator.transaction.ProducerIdManager)
    [2017-12-3115:52:21,782] INFO [TransactionCoordinator id=0] Starting up. (kafka.coordinator.transaction.TransactionCoordinator)
    [2017-12-3115:52:21,784] INFO [Transaction Marker Channel Manager0]:Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
    [2017-12-3115:52:21,797] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)
    [2017-12-3115:52:21,889] INFO Creating/brokers/ids/0 (isit secure? false) (kafka.utils.ZKCheckedEphemeral)
    [2017-12-3115:52:21,895] INFO Result of znode creation is:OK (kafka.utils.ZKCheckedEphemeral)
    [2017-12-3115:52:21,897] INFO Registered broker0at path/brokers/ids/0with addresses:EndPoint(192.168.0.104,9092,ListenerName(PLAINTEXT),PLAINTEXT) (kafka.utils.ZkUtils)
    [2017-12-3115:52:21,900] WARN No meta.properties file under dir/tmp/kafka-logs/meta.properties (kafka.server.BrokerMetadataCheckpoint)
    [2017-12-3115:52:21,920] INFO Kafka version:1.0.0 (org.apache.kafka.common.utils.AppInfoParser)
    [2017-12-3115:52:21,920] INFO Kafka commitId:aaa7af6d4a11b29d (org.apache.kafka.common.utils.AppInfoParser)
    [2017-12-3115:52:21,922] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)

    Download Kafka Tool

    Conclusion :

    In this Apache Kafka Tutorial, we have successfully learnt to install Apache Kafka on Mac and start Kafka along with Zookeeper.

    Install Homebrew

    Homebrew is a great little package manager for OS X. If you haven't already, installing it is pretty easy:

    Install Java

    Download and install Java: https://java.com/en/download/help/mac_install.xml

    Install Kafka

    This installs Apache Kafka, and dependency zookeeper if it's not already installed:

    Starting Kafka

    Kafka Tool

    Starting zookeeper and default kafka server:

    • Properties: /usr/local/etc/kafka
    • Logs: /usr/local/var/log/kafka
    • Data: /usr/local/var/lib/kafka-logs

    Links

    Kafka Download Linux

    Mac

    Download Kafka Mac Download

    • [Apache Kafka] (https://kafka.apache.org/)
    • [Kafka and Spring boot] (https://spring.io/blog/2015/04/15/using-apache-kafka-for-integration-and-data-processing-pipelines-with-spring)