Archives | DataScienceTribe

2023

02

Difference between fit, transform, fit_transform, predict, and predict_proba in a sklearn pipeline

What is pipeline in scikit-learnPipeline in scikit-learn is a utility class that helps to assemble several steps of an M

2023-02-02 data science

01

Spark SQL - Collecting Columns into Lists after Groupby

In Spark SQL, you may want to collect the values of one or more columns into lists after grouping the data by one or mor

2023-02-01 data engineering

spark pyspark

01

25

How to define mulitple endpoints in one API using AWS API gateway

By default, when we create API using the AWS API Gateway service, the default resource generated is the root path “/“.An

2023-01-25 data science

aws api gateway

22

In PySpark how to filter dataframe column using unique values from another dataframe

Here is one common task in PySpark: how to filter one dataframe column are from unique values from anther dataframe? Met

2023-01-22 data engineering

spark pyspark

20

How to generate access token in Databricks

In Databricks, personal access tokens can be used for secure authentication to the Databricks API instead of passwords.

2023-01-20 data engineering

databricks

18

FAISS index and normalization

Previously, we have discussed how to implement a real time semantic search using sentence transformer and FAISS.real tim

2023-01-18 data science

semantic search FAISS

15

The fanaticism and reality of Web3

In the past two years, the concept of “Web3” has become increasingly popular in the IT industry. In the past 2022, web3

2023-01-15 web3

web3

14

Real time in memory semantic search with deep learning embedding and FAISS

We use sentence transformer to encode short texts, and then index the results using in memory search engine FAISS;Togeht

2023-01-14 data science

deep learning semantic search vector search FAISS

2022

12

19

Feature reduction and visualization using autoencoder with Pytorch

In this notebook, we are going to use autoencoder architecture in Pytorch to reduce feature dimensions and visualiations

2022-12-19 machine learning

pytorch deep learning autoencoder

15

Revolutionizing Robotics, How RT-1 is Paving the Way for Intelligent Machines

Great progress of robotics and learning from language models. Researchers at OpenAI have developed the Robotics Transfor

2022-12-15 machine learning

news robotics

14

A list of poupular but hard leetcode problems for interview

Here is the list most popular but hard interview questions that according to leetcode. List of problems and solutions fi

2022-12-14 algorithm

leetcode interview questions

12

A custom scikit-learn transformer example with parameter input and avoid possible None error

A customer scikit-learn transfomer example with parameter.It takes a pandas DataFrame with a column called input as inpu

2022-12-12 data science

pandas scikit-learn